Benchmark Almanac

LegalBench

Claude Fable 5 88.6% leads since Jun 2026 Not in model cards

Models do 162 legal reasoning tasks written by legal professionals, covering six types of legal reasoning such as issue spotting and rule application.

Vals AI runs it on 151 models. Claude Fable 5 led at 88.56% on 2026-10-13.

In the authors’ words

The advent of large language models (LLMs) and their adoption by the legal community has given rise to the question: what types of legal reasoning can LLMs perform? To enable greater study of this question, we present LegalBench: a collaboratively constructed legal reasoning benchmark consisting of 162 tasks covering six different types of legal reasoning. LegalBench was built through an interdisciplinary process, in which we collected tasks designed and hand-crafted by legal professionals. Because these subject matter experts took a leading role in construction, tasks either measure legal reasoning capabilities that are practically useful, or measure reasoning skills that lawyers find interesting. To enable cross-disciplinary conversations about LLMs in the law, we additionally show how popular legal frameworks for describing legal reasoning – which distinguish between its many forms – correspond to LegalBench tasks, thus giving lawyers and LLM developers a common vocabulary. This paper describes LegalBench, presents an empirical evaluation of 20 open-source and commercial LLMs, and illustrates the types of research explorations LegalBench enables.

Abstract of LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models (arXiv:2308.11462, CC0)
Made by Stanford University (Hazy Research) and collaborators Released Aug 2023 Law All tasks public Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
HazyResearch/legalbench 653 stars Paper Dataset Leaderboard hazyresearch.stanford.edu
139
Entries on its own leaderboard, 138 models we track

Claude Fable 5 leads with 88.6%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

139 entries. Scroll sideways for the rest.

  1. 88.6 Claude Fable 5
  2. 88.5 Claude Fable 5.1
  3. 88.3 Gemini 4 Argon
  4. 87.4 Muse Wonder Full
  5. 87.4 Gemini 3.1 Pro
  6. 87.3 Gemini 3.7 Flash
  7. 87.0 Gemini 3 Pro
  8. 87.0 Gemini 3.8 Flash
  9. 87.0 GPT-5.6 Sol
  10. 87.0 Claude Opus 5
  11. 86.9 Gemini 3 Flash
  12. 86.7 Gemini 3.6 Flash
  13. 86.5 GPT-5.5
  14. 86.3 Grok 4.6
  15. 86.2 GLM Legal
  16. 86.2 Kimi K3
  17. 86.0 GPT-5.4
  18. 86.0 GPT-5
  19. 86.0 Grok 4.5
  20. 85.7 GPT-5.1
  21. 85.4 MiniMax-M3
  22. 85.3 Claude Opus 4.6
  23. 85.3 Muse Spark 1.2
  24. 85.3 Claude Opus 4.7
  25. 85.1 GPT-5.6 Terra
  26. 85.1 Qwen 3.5 Plus (hosted 397B-A17B)
  27. 85.0 Muse Spark 1.1
  28. 84.9 Qwen3.7-Max
  29. 84.8 GLM-5.3
  30. 84.7 Kimi K2.6
  31. 84.6 Claude Opus 4.5
  32. 84.5 Grok 4.3
  33. 84.4 GLM-5.1
  34. 84.4 Grok 4.7
  35. 84.3 Gemini 2.5 Pro
  36. 84.3 Qwen 3.5 Flash (hosted 35B-A3B)
  37. 84.2 Qwen 3.6 Plus
  38. 84.2 Muse Spark
  39. 84.1 Claude Sonnet 4.5
  40. 84.1 GLM-5.2
  41. 84.1 Gemini 3.5 Flash-Lite
  42. 84.1 GLM-5
  43. 84.0 GPT-5.6 Luna
  44. 84.0 MiniMax-M2.7
  45. 83.9 GLM-5.3-Flash
  46. 83.9 Claude Sonnet 5
  47. 83.8 Gemini 2.5 Flash
  48. 83.8 o3
  49. 83.8 Gemini 3.1 Flash-Lite
  50. 83.8 Tencent Hy4 preview
  51. 83.6 Qwen 3.8 Max
  52. 83.6 Gemini 3.5 Flash
  53. 83.6 Claude Opus 4.8
  54. 83.5 Claude Opus 4.1
  55. 83.4 GLM-4.7
  56. 83.3 DeepSeek V4.1 Flash
  57. 83.2 Grok 4
  58. 83.1 Grok-3 mini
  59. 83.1 GPT-4.1
  60. 83.1 Claude Opus 4
  61. 83.1 Mercury 2.5
  62. 83.0 Inkling-Small
  63. 83.0 Claude Sonnet 4
  64. 83.0 Inkling
  65. 82.8 GPT-5.2
  66. 82.6 Gemini 2.5 Flash
  67. 82.6 Grok 3
  68. 82.5 Claude 3.7 Sonnet
  69. 82.5 Grok 4.1 Fast
  70. 82.4 Qwen 3.8 27B
  71. 82.4 DeepSeek V4 Pro 0813
  72. 82.2 GPT-4o
  73. 82.1 Claude Sonnet 4.6
  74. 82.1 Nemotron 3 Ultra
  75. 82.0 Gemini 2.5 Flash-Lite
  76. 81.9 Qwen3-Max
  77. 81.8 GPT-5 mini
  78. 81.5 Kimi K2
  79. 81.2 Claude Haiku 4.5
  80. 80.8 DeepSeek-V3
  81. 80.6 Grok 4 Fast
  82. 80.5 GPT-4 Turbo
  83. 80.4 o1
  84. 80.3 DeepSeek-V4-Pro
  85. 80.2 Kimi K2 Thinking
  86. 80.2 Qwen3-235B-A22B
  87. 80.0 MiniMax-M2.5
  88. 79.7 Cohere Command A
  89. 79.7 Ling-3.0-flash
  90. 79.6 GLM-4.6
  91. 79.4 Qwen2.5-72B
  92. 79.2 o4-mini
  93. 79.1 Mistral Large 3
  94. 78.9 MiMo-V2.5
  95. 78.4 Gemini 2.0 Flash
  96. 78.0 GPT-4.1 mini
  97. 77.9 GPT-5.4 Nano
  98. 77.8 Llama 4 Maverick
  99. 77.7 Grok 4.20
  100. 77.7 DeepSeek V4 Flash 0731
  101. 77.2 Llama 3.3 70B
  102. 77.1 MiMo-V2.5-Pro
  103. 76.1 DeepSeek-V3.2
  104. 75.9 gpt-oss-120b
  105. 75.6 GLM-4.5
  106. 75.5 MiniMax-M2.1
  107. 75.1 Laguna M.1
  108. 74.2 Jamba 1.5 Large
  109. 72.0 Llama 4 Scout
  110. 72.0 Jamba 1.6 Large
  111. 71.5 o3-mini
  112. 71.0 Laguna XS.2
  113. 70.8 gpt-oss-20b
  114. 70.3 Claude 3.5 Haiku
  115. 70.3 Gemini 1.0 Pro
  116. 69.7 Jamba 1.6 Mini
  117. 69.6 Qwen2.5-7B
  118. 69.2 Mistral Small 3.1
  119. 69.1 Gemini 1.5 Pro
  120. 68.7 Ling-3.0-flash-Fin
  121. 68.3 Command R+
  122. 68.1 Gemma 2 9B
  123. 68.0 Gemma 2 27B
  124. 67.3 DeepSeek-R1
  125. 66.6 Jamba 1.5 Mini
  126. 64.4 GPT 3.5
  127. 63.2 Gemini 1.5 Flash
  128. 62.4 Llama 2-70B
  129. 62.0 Mistral Medium 3.1
  130. 61.4 Command A+
  131. 61.1 GPT-4.1 nano
  132. 55.8 Mixtral 8x7B
  133. 54.8 Magistral Medium 1.2
  134. 53.8 Mistral 7B v0.1
  135. 53.7 Llama 2-13B
  136. 51.9 Llama 2-7B
  137. 50.1 GPT-5 nano
  138. 40.0 Magistral Small 1.2
  139. 33.0 Command R (03-2024)
  1. 1 Claude Fable 5 88.6%
  2. 2 Claude Fable 5.1 88.5%
  3. 3 Gemini 4 Argon 88.3%
  4. 4 Muse Wonder Full 87.4%
  5. 5 Gemini 3.1 Pro 87.4%
  6. 6 Gemini 3.7 Flash 87.3%
  7. 7 Gemini 3 Pro 87%
  8. 8 Gemini 3.8 Flash 87%
  9. 9 GPT-5.6 Sol 87%
  10. 10 Claude Opus 5 87%
  11. 11 Gemini 3 Flash 86.9%
  12. 12 Gemini 3.6 Flash 86.7%
  13. 13 GPT-5.5 86.5%
  14. 14 Grok 4.6 86.3%
  15. 15 GLM Legal 86.2%
  16. 16 Kimi K3 86.2%
  17. 17 GPT-5.4 xhigh 86%
  18. 18 GPT-5 86%
  19. 19 Grok 4.5 86%
  20. 20 GPT-5.1 85.7%
  21. 21 MiniMax-M3 85.4%
  22. 22 Claude Opus 4.6 thinking 85.3%
  23. 23 Muse Spark 1.2 85.3%
  24. 24 Claude Opus 4.7 85.3%
  25. 25 GPT-5.6 Terra 85.1%
  26. 26 Qwen 3.5 Plus (hosted 397B-A17B) thinking 85.1%
  27. 27 Muse Spark 1.1 85%
  28. 28 Qwen3.7-Max 84.9%
  29. 29 GLM-5.3 84.8%
  30. 30 Kimi K2.6 84.7%
  31. 31 Claude Opus 4.5 thinking 84.6%
  32. 32 Grok 4.3 84.5%
  33. 33 GLM-5.1 84.4%
  34. 34 Grok 4.7 84.4%
  35. 35 Gemini 2.5 Pro Jun 2025 84.3%
  36. 36 Qwen 3.5 Flash (hosted 35B-A3B) 84.3%
  37. 37 Qwen 3.6 Plus 84.2%
  38. 38 Muse Spark 84.2%
  39. 39 Claude Sonnet 4.5 thinking 84.1%
  40. 40 GLM-5.2 84.1%
  41. 41 Gemini 3.5 Flash-Lite 84.1%
  42. 42 GLM-5 thinking 84.1%
  43. 43 GPT-5.6 Luna 84%
  44. 44 MiniMax-M2.7 84%
  45. 45 GLM-5.3-Flash 83.9%
  46. 46 Claude Sonnet 5 83.9%
  47. 47 Gemini 2.5 Flash Apr 2025, non-thinking 83.8%
  48. 48 o3 83.8%
  49. 49 Gemini 3.1 Flash-Lite 83.8%
  50. 50 Tencent Hy4 preview 83.8%
  51. 51 Qwen 3.8 Max 83.6%
  52. 52 Gemini 3.5 Flash 83.6%
  53. 53 Claude Opus 4.8 83.6%
  54. 54 Claude Opus 4.1 non-thinking 83.5%
  55. 55 GLM-4.7 83.4%
  56. 56 DeepSeek V4.1 Flash 83.3%
  57. 57 Grok 4 83.2%
  58. 58 Grok-3 mini reasoning 83.1%
  59. 59 GPT-4.1 83.1%
  60. 60 Claude Opus 4 non-thinking 83.1%
  61. 61 Mercury 2.5 83.1%
  62. 62 Inkling-Small 83%
  63. 63 Claude Sonnet 4 non-thinking 83%
  64. 64 Inkling 83%
  65. 65 GPT-5.2 82.8%
  66. 66 Gemini 2.5 Flash Sep 2025, thinking 82.6%
  67. 67 Grok 3 82.6%
  68. 68 Claude 3.7 Sonnet thinking 82.5%
  69. 69 Grok 4.1 Fast reasoning 82.5%
  70. 70 Qwen 3.8 27B 82.4%
  71. 71 DeepSeek V4 Pro 0813 82.4%
  72. 72 GPT-4o Aug 2024 82.2%
  73. 73 Claude Sonnet 4.6 82.1%
  74. 74 Nemotron 3 Ultra 82.1%
  75. 75 Gemini 2.5 Flash-Lite Sep 2025, thinking 82%
  76. 76 Qwen3-Max 81.9%
  77. 77 GPT-5 mini 81.8%
  78. 78 Kimi K2 Jul 2025 81.5%
  79. 79 Claude Haiku 4.5 thinking 81.2%
  80. 80 DeepSeek-V3 Mar 2025 80.8%
  81. 81 Grok 4 Fast reasoning 80.6%
  82. 82 GPT-4 Turbo Apr 2024 80.5%
  83. 83 o1 80.4%
  84. 84 DeepSeek-V4-Pro 80.3%
  85. 85 Kimi K2 Thinking 80.2%
  86. 86 Qwen3-235B-A22B 80.2%
  87. 87 MiniMax-M2.5 80%
  88. 88 Cohere Command A 79.7%
  89. 89 Ling-3.0-flash 79.7%
  90. 90 GLM-4.6 79.6%
  91. 91 Qwen2.5-72B 79.4%
  92. 92 o4-mini 79.2%
  93. 93 Mistral Large 3 79.1%
  94. 94 MiMo-V2.5 78.9%
  95. 95 Gemini 2.0 Flash Feb 2025 78.4%
  96. 96 GPT-4.1 mini 78%
  97. 97 GPT-5.4 Nano 77.9%
  98. 98 Llama 4 Maverick 77.8%
  99. 99 Grok 4.20 reasoning 77.7%
  100. 100 DeepSeek V4 Flash 0731 77.7%
  101. 101 Llama 3.3 70B 77.2%
  102. 102 MiMo-V2.5-Pro 77.1%
  103. 103 DeepSeek-V3.2 thinking 76.1%
  104. 104 gpt-oss-120b 75.9%
  105. 105 GLM-4.5 75.6%
  106. 106 MiniMax-M2.1 75.5%
  107. 107 Laguna M.1 75.1%
  108. 108 Jamba 1.5 Large 74.2%
  109. 109 Llama 4 Scout 72%
  110. 110 Jamba 1.6 Large 72%
  111. 111 o3-mini 71.5%
  112. 112 Laguna XS.2 71%
  113. 113 gpt-oss-20b 70.8%
  114. 114 Claude 3.5 Haiku 70.3%
  115. 115 Gemini 1.0 Pro 70.3%
  116. 116 Jamba 1.6 Mini 69.7%
  117. 117 Qwen2.5-7B 69.6%
  118. 118 Mistral Small 3.1 69.2%
  119. 119 Gemini 1.5 Pro Sep 2024 69.1%
  120. 120 Ling-3.0-flash-Fin 68.7%
  121. 121 Command R+ 68.3%
  122. 122 Gemma 2 9B 68.1%
  123. 123 Gemma 2 27B 68%
  124. 124 DeepSeek-R1 May 2025 67.3%
  125. 125 Jamba 1.5 Mini 66.6%
  126. 126 GPT 3.5 64.4%
  127. 127 Gemini 1.5 Flash Sep 2024 63.2%
  128. 128 Llama 2-70B 62.4%
  129. 129 Mistral Medium 3.1 62%
  130. 130 Command A+ 61.4%
  131. 131 GPT-4.1 nano 61.1%
  132. 132 Mixtral 8x7B 55.8%
  133. 133 Magistral Medium 1.2 54.8%
  134. 134 Mistral 7B v0.1 53.8%
  135. 135 Llama 2-13B 53.7%
  136. 136 Llama 2-7B 51.9%
  137. 137 GPT-5 nano 50.1%
  138. 138 Magistral Small 1.2 40%
  139. 139 Command R (03-2024) 33%
#EntryScoreSettingDateSource
1 Claude Fable 5 88.6% Jun 2026 Official leaderboard
2 Claude Fable 5.1 88.5% Sep 2026 Official leaderboard
3 Gemini 4 Argon 88.3% Sep 2026 Official leaderboard
4 Muse Wonder Full 87.4% Oct 2026 Official leaderboard
5 Gemini 3.1 Pro 87.4% Feb 2026 Official leaderboard
6 Gemini 3.7 Flash 87.3% Aug 2026 Official leaderboard
7 Gemini 3 Pro 87% Nov 2025 Official leaderboard
8 Gemini 3.8 Flash 87% Sep 2026 Official leaderboard
9 GPT-5.6 Sol 87% Jul 2026 Official leaderboard
10 Claude Opus 5 87% Jul 2026 Official leaderboard
11 Gemini 3 Flash 86.9% Dec 2025 Official leaderboard
12 Gemini 3.6 Flash 86.7% Jul 2026 Official leaderboard
13 GPT-5.5 86.5% Apr 2026 Official leaderboard
14 Grok 4.6 86.3% Aug 2026 Official leaderboard
15 GLM Legal 86.2% Oct 2026 Official leaderboard
16 Kimi K3 86.2% Jul 2026 Official leaderboard
17 GPT-5.4 86% xhigh Mar 2026 Official leaderboard
18 GPT-5 86% Aug 2025 Official leaderboard
19 Grok 4.5 86% Jul 2026 Official leaderboard
20 GPT-5.1 85.7% Nov 2025 Official leaderboard
21 MiniMax-M3 85.4% Jun 2026 Official leaderboard
22 Claude Opus 4.6 85.3% thinking Feb 2026 Official leaderboard
23 Muse Spark 1.2 85.3% Aug 2026 Official leaderboard
24 Claude Opus 4.7 85.3% Apr 2026 Official leaderboard
25 GPT-5.6 Terra 85.1% Jul 2026 Official leaderboard
26 Qwen 3.5 Plus (hosted 397B-A17B) 85.1% thinking Feb 2026 Official leaderboard
27 Muse Spark 1.1 85% Jul 2026 Official leaderboard
28 Qwen3.7-Max 84.9% May 2026 Official leaderboard
29 GLM-5.3 84.8% Aug 2026 Official leaderboard
30 Kimi K2.6 84.7% Apr 2026 Official leaderboard
31 Claude Opus 4.5 84.6% thinking Nov 2025 Official leaderboard
32 Grok 4.3 84.5% Apr 2026 Official leaderboard
33 GLM-5.1 84.4% Apr 2026 Official leaderboard
34 Grok 4.7 84.4% Sep 2026 Official leaderboard
35 Gemini 2.5 Pro Jun 2025 84.3% Jun 2025 Official leaderboard
36 Qwen 3.5 Flash (hosted 35B-A3B) 84.3% Feb 2026 Official leaderboard
37 Qwen 3.6 Plus 84.2% Mar 2026 Official leaderboard
38 Muse Spark 84.2% Apr 2026 Official leaderboard
39 Claude Sonnet 4.5 84.1% thinking Sep 2025 Official leaderboard
40 GLM-5.2 84.1% Jun 2026 Official leaderboard
41 Gemini 3.5 Flash-Lite 84.1% Jul 2026 Official leaderboard
42 GLM-5 84.1% thinking Feb 2026 Official leaderboard
43 GPT-5.6 Luna 84% Jul 2026 Official leaderboard
44 MiniMax-M2.7 84% Mar 2026 Official leaderboard
45 GLM-5.3-Flash 83.9% Aug 2026 Official leaderboard
46 Claude Sonnet 5 83.9% Jun 2026 Official leaderboard
47 Gemini 2.5 Flash Apr 2025 83.8% non-thinking Apr 2025 Official leaderboard
48 o3 83.8% Apr 2025 Official leaderboard
49 Gemini 3.1 Flash-Lite 83.8% Mar 2026 Official leaderboard
50 Tencent Hy4 preview 83.8% Aug 2026 Official leaderboard
51 Qwen 3.8 Max 83.6% Aug 2026 Official leaderboard
52 Gemini 3.5 Flash 83.6% May 2026 Official leaderboard
53 Claude Opus 4.8 83.6% May 2026 Official leaderboard
54 Claude Opus 4.1 83.5% non-thinking Aug 2025 Official leaderboard
55 GLM-4.7 83.4% Dec 2025 Official leaderboard
56 DeepSeek V4.1 Flash 83.3% Sep 2026 Official leaderboard
57 Grok 4 83.2% Jul 2025 Official leaderboard
58 Grok-3 mini 83.1% reasoning Apr 2025 Official leaderboard
59 GPT-4.1 83.1% Apr 2025 Official leaderboard
60 Claude Opus 4 83.1% non-thinking May 2025 Official leaderboard
61 Mercury 2.5 83.1% Sep 2026 Official leaderboard
62 Inkling-Small 83% Jul 2026 Official leaderboard
63 Claude Sonnet 4 83% non-thinking May 2025 Official leaderboard
64 Inkling 83% Jul 2026 Official leaderboard
65 GPT-5.2 82.8% Dec 2025 Official leaderboard
66 Gemini 2.5 Flash Sep 2025 82.6% thinking Sep 2025 Official leaderboard
67 Grok 3 82.6% Apr 2025 Official leaderboard
68 Claude 3.7 Sonnet 82.5% thinking Feb 2025 Official leaderboard
69 Grok 4.1 Fast 82.5% reasoning Nov 2025 Official leaderboard
70 Qwen 3.8 27B 82.4% Aug 2026 Official leaderboard
71 DeepSeek V4 Pro 0813 82.4% Aug 2026 Official leaderboard
72 GPT-4o Aug 2024 82.2% Aug 2024 Official leaderboard
73 Claude Sonnet 4.6 82.1% Feb 2026 Official leaderboard
74 Nemotron 3 Ultra 82.1% Jun 2026 Official leaderboard
75 Gemini 2.5 Flash-Lite Sep 2025 82% thinking Sep 2025 Official leaderboard
76 Qwen3-Max 81.9% Sep 2025 Official leaderboard
77 GPT-5 mini 81.8% Aug 2025 Official leaderboard
78 Kimi K2 Jul 2025 81.5% Jul 2025 Official leaderboard
79 Claude Haiku 4.5 81.2% thinking Oct 2025 Official leaderboard
80 DeepSeek-V3 Mar 2025 80.8% Mar 2025 Official leaderboard
81 Grok 4 Fast 80.6% reasoning Sep 2025 Official leaderboard
82 GPT-4 Turbo Apr 2024 80.5% Apr 2024 Official leaderboard
83 o1 80.4% Dec 2024 Official leaderboard
84 DeepSeek-V4-Pro 80.3% Apr 2026 Official leaderboard
85 Kimi K2 Thinking 80.2% Nov 2025 Official leaderboard
86 Qwen3-235B-A22B 80.2% Apr 2025 Official leaderboard
87 MiniMax-M2.5 80% Feb 2026 Official leaderboard
88 Cohere Command A 79.7% Mar 2025 Official leaderboard
89 Ling-3.0-flash 79.7% Aug 2026 Official leaderboard
90 GLM-4.6 79.6% Sep 2025 Official leaderboard
91 Qwen2.5-72B 79.4% Sep 2024 Official leaderboard
92 o4-mini 79.2% Apr 2025 Official leaderboard
93 Mistral Large 3 79.1% Dec 2025 Official leaderboard
94 MiMo-V2.5 78.9% Apr 2026 Official leaderboard
95 Gemini 2.0 Flash Feb 2025 78.4% Feb 2025 Official leaderboard
96 GPT-4.1 mini 78% Apr 2025 Official leaderboard
97 GPT-5.4 Nano 77.9% Mar 2026 Official leaderboard
98 Llama 4 Maverick 77.8% Apr 2025 Official leaderboard
99 Grok 4.20 77.7% reasoning Feb 2026 Official leaderboard
100 DeepSeek V4 Flash 0731 77.7% Jul 2026 Official leaderboard
101 Llama 3.3 70B 77.2% Dec 2024 Official leaderboard
102 MiMo-V2.5-Pro 77.1% Apr 2026 Official leaderboard
103 DeepSeek-V3.2 76.1% thinking Dec 2025 Official leaderboard
104 gpt-oss-120b 75.9% Aug 2025 Official leaderboard
105 GLM-4.5 75.6% Aug 2025 Official leaderboard
106 MiniMax-M2.1 75.5% Dec 2025 Official leaderboard
107 Laguna M.1 75.1% Apr 2026 Official leaderboard
108 Jamba 1.5 Large 74.2% Aug 2024 Official leaderboard
109 Llama 4 Scout 72% Apr 2025 Official leaderboard
110 Jamba 1.6 Large 72% Mar 2025 Official leaderboard
111 o3-mini 71.5% Jan 2025 Official leaderboard
112 Laguna XS.2 71% Apr 2026 Official leaderboard
113 gpt-oss-20b 70.8% Aug 2025 Official leaderboard
114 Claude 3.5 Haiku 70.3% Oct 2024 Official leaderboard
115 Gemini 1.0 Pro 70.3% Dec 2023 Official leaderboard
116 Jamba 1.6 Mini 69.7% Mar 2025 Official leaderboard
117 Qwen2.5-7B 69.6% Sep 2024 Official leaderboard
118 Mistral Small 3.1 69.2% Mar 2025 Official leaderboard
119 Gemini 1.5 Pro Sep 2024 69.1% Sep 2024 Official leaderboard
120 Ling-3.0-flash-Fin 68.7% Sep 2026 Official leaderboard
121 Command R+ 68.3% Aug 2024 Official leaderboard
122 Gemma 2 9B 68.1% Jun 2024 Official leaderboard
123 Gemma 2 27B 68% Jun 2024 Official leaderboard
124 DeepSeek-R1 May 2025 67.3% May 2025 Official leaderboard
125 Jamba 1.5 Mini 66.6% Aug 2024 Official leaderboard
126 GPT 3.5 64.4% Official leaderboard
127 Gemini 1.5 Flash Sep 2024 63.2% Aug 2024 Official leaderboard
128 Llama 2-70B 62.4% Jul 2023 Official leaderboard
129 Mistral Medium 3.1 62% Aug 2025 Official leaderboard
130 Command A+ 61.4% May 2026 Official leaderboard
131 GPT-4.1 nano 61.1% Apr 2025 Official leaderboard
132 Mixtral 8x7B 55.8% Dec 2023 Official leaderboard
133 Magistral Medium 1.2 54.8% Sep 2025 Official leaderboard
134 Mistral 7B v0.1 53.8% Sep 2023 Official leaderboard
135 Llama 2-13B 53.7% Jul 2023 Official leaderboard
136 Llama 2-7B 51.9% Jul 2023 Official leaderboard
137 GPT-5 nano 50.1% Aug 2025 Official leaderboard
138 Magistral Small 1.2 40% Sep 2025 Official leaderboard
139 Command R (03-2024) 33% Mar 2024 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 4.5
non-thinking thinking
84.6%
Claude Sonnet 4
thinking
83%
Gemini 2.5 Flash
non-thinking
82.6%
Claude 3.7 Sonnet
non-thinking thinking
82.5%
Grok 4.1 Fast
non-reasoni… reasoning
82.5%
Gemini 2.5 Flash-Lite
non-thinking thinking
82%
Grok 4 Fast
non-reasoni… reasoning
80.6%
DeepSeek-V3.2
non-thinking thinking
76.1%
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5 88.6%
Claude Fable 5.1 88.5%
Gemini 4 Argon 88.3%
Muse Wonder Full 87.4%
Gemini 3.1 Pro 87.4%
Gemini 3.7 Flash 87.3%
Gemini 3 Pro 87%
Gemini 3.8 Flash 87%
GPT-5.6 Sol 87%
Claude Opus 5 87%
Gemini 3 Flash 86.9%
Gemini 3.6 Flash 86.7%
GPT-5.5 86.5%
Grok 4.6 86.3%
GLM Legal 86.2%
Kimi K3 86.2%
GPT-5.4 xhigh 86%
GPT-5 86%
Grok 4.5 86%
GPT-5.1 85.7%
MiniMax-M3 85.4%
Claude Opus 4.6 thinking 85.3%
Muse Spark 1.2 85.3%
Claude Opus 4.7 85.3%
GPT-5.6 Terra 85.1%
Qwen 3.5 Plus (hosted 397B-A17B) thinking 85.1%
Muse Spark 1.1 85%
Qwen3.7-Max 84.9%
GLM-5.3 84.8%
Kimi K2.6 84.7%
Claude Opus 4.5 thinking 84.6%
Grok 4.3 84.5%
GLM-5.1 84.4%
Grok 4.7 84.4%
Gemini 2.5 Pro Jun 2025 84.3%
Qwen 3.5 Flash (hosted 35B-A3B) 84.3%
Qwen 3.6 Plus 84.2%
Muse Spark 84.2%
Claude Sonnet 4.5 thinking 84.1%
GLM-5.2 84.1%
Gemini 3.5 Flash-Lite 84.1%
GLM-5 thinking 84.1%
GPT-5.6 Luna 84%
MiniMax-M2.7 84%
GLM-5.3-Flash 83.9%
Claude Sonnet 5 83.9%
Gemini 2.5 Flash Apr 2025 non-thinking 83.8%
o3 83.8%
Gemini 3.1 Flash-Lite 83.8%
Tencent Hy4 preview 83.8%
Qwen 3.8 Max 83.6%
Gemini 3.5 Flash 83.6%
Claude Opus 4.8 83.6%
Claude Opus 4.1 non-thinking 83.5%
GLM-4.7 83.4%
DeepSeek V4.1 Flash 83.3%
Grok 4 83.2%
Grok-3 mini reasoning 83.1%
GPT-4.1 83.1%
Claude Opus 4 non-thinking 83.1%
Mercury 2.5 83.1%
Inkling-Small 83%
Claude Sonnet 4 non-thinking 83%
Inkling 83%
Claude Opus 4.5 non-thinking 82.8%
GPT-5.2 82.8%
Gemini 2.5 Flash Sep 2025 thinking 82.6%
Grok 3 82.6%
Claude 3.7 Sonnet thinking 82.5%
Gemini 2.5 Flash Sep 2025 non-thinking 82.5%
Grok 4.1 Fast reasoning 82.5%
Qwen 3.8 27B 82.4%
DeepSeek V4 Pro 0813 82.4%
GPT-4o Aug 2024 82.2%
Claude Sonnet 4.6 82.1%
Nemotron 3 Ultra 82.1%
Claude Sonnet 4 thinking 82.1%
Gemini 2.5 Flash-Lite Sep 2025 thinking 82%
Grok-3 mini reasoning 81.9%
Qwen3-Max 81.9%
GPT-5 mini 81.8%
Kimi K2 Jul 2025 81.5%
Claude Haiku 4.5 thinking 81.2%
DeepSeek-V3 Mar 2025 80.8%
Grok 4 Fast reasoning 80.6%
GPT-4 Turbo Apr 2024 80.5%
o1 80.4%
Qwen3-Max 80.3%
DeepSeek-V4-Pro 80.3%
Kimi K2 Thinking 80.2%
Qwen3-235B-A22B 80.2%
GPT-4o Aug 2024 80.1%
Claude 3.7 Sonnet non-thinking 80%
MiniMax-M2.5 80%
Cohere Command A 79.7%
Ling-3.0-flash 79.7%
GLM-4.6 79.6%
Qwen2.5-72B 79.4%
o4-mini 79.2%
Mistral Large 3 79.1%
Grok 4.1 Fast non-reasoning 79.1%
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 79%
MiMo-V2.5 78.9%
Grok 4 Fast non-reasoning 78.4%
Gemini 2.0 Flash Feb 2025 78.4%
GPT-4.1 mini 78%
GPT-5.4 Nano 77.9%
Llama 4 Maverick 77.8%
Grok 4.20 reasoning 77.7%
DeepSeek-V3 Mar 2025 77.7%
DeepSeek V4 Flash 0731 77.7%
Llama 3.3 70B 77.2%
MiMo-V2.5-Pro 77.1%
DeepSeek-V3.2 thinking 76.1%
gpt-oss-120b 75.9%
GLM-4.5 75.6%
MiniMax-M2.1 75.5%
Laguna M.1 75.1%
Jamba 1.5 Large 74.2%
Llama 4 Scout 72%
Jamba 1.6 Large 72%
o3-mini 71.5%
Laguna XS.2 71%
gpt-oss-20b 70.8%
Claude 3.5 Haiku 70.3%
Gemini 1.0 Pro 70.3%
Jamba 1.6 Mini 69.7%
Qwen2.5-7B 69.6%
Mistral Small 3.1 69.2%
Gemini 1.5 Pro Sep 2024 69.1%
Ling-3.0-flash-Fin 68.7%
Command R+ 68.3%
Gemma 2 9B 68.1%
Gemma 2 27B 68%
DeepSeek-R1 May 2025 67.3%
Jamba 1.5 Mini 66.6%
DeepSeek-V3.2 non-thinking 66%
Gemini 1.5 Flash Sep 2024 63.2%
Llama 2-70B 62.4%
Mistral Medium 3.1 62%
Command A+ 61.4%
GPT-4.1 nano 61.1%
Mixtral 8x7B 55.8%
Magistral Medium 1.2 54.8%
Mistral 7B v0.1 53.8%
Llama 2-13B 53.7%
Llama 2-7B 51.9%
GPT-5 nano 50.1%
Magistral Small 1.2 40%
Command R (03-2024) 33%

13 models have held number one since its release in Aug 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta Google DeepMind OpenAI Anthropic xAI
Own leaderboard Passed 85% in Aug 2025, by GPT-5
Claude Fable 5 88.6%

The best score passed 85% in Aug 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Fable 5 Gemini 3 Pro Gemini 2.5 Flash GPT-4o Gemini 1.0 Pro Llama 2-70B 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Fable 5 GPT-4o

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 140 in Law, finance and medicine
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%
  • PRBench In use Model answers 1,100 expert-written law and finance questions. LLM judge grades against weighted expert rubrics of 10-30 criteria each. Muse Spark 1.3 61.6%