Benchmark Almanac

ACCoRD

Claude Opus 5.5 86.5 leads since Sep 2026 Not in model cards

Models state probabilities and preferences on conceptual questions, and the benchmark checks 567 expert-approved logical consistency constraints such as conjunction rules and order invariance.

A component of the Conceptual Reasoning Index. The full dataset has 13,853 model-generated constraints in 18 types. No paper yet.

Made by Redwood Research (Cooper, Kastner, Oesterheld et al.) Released Aug 2026 Other All tasks public Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Prediction Probabilities for events that have not happened yet.
casparoe/accord_public 2 stars Leaderboard conceptualreasoning.ai
152
Entries on its own leaderboard, 150 models we track

Claude Opus 5.5 leads with 86.5

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

152 entries. Scroll sideways for the rest.

  1. 86.5 Claude Opus 5.5
  2. 85.1 Claude Sonnet 5.5
  3. 85.0 Claude Fable 5
  4. 82.4 GPT-6 Sol
  5. 82.0 Claude Opus 5
  6. 81.7 GPT-6 Astra
  7. 81.5 Claude Haiku 5.5
  8. 81.0 Claude Fable 5.1
  9. 80.9 GPT-6.1 Sol
  10. 80.5 GPT-5.5
  11. 80.4 Claude Opus 4.6
  12. 80.0 GPT-5.4
  13. 79.8 Claude Opus 4.7
  14. 79.3 GPT-5.2
  15. 79.1 Claude Opus 4.5
  16. 78.9 GPT-5.6 Sol Pro
  17. 78.4 Muse Spark 1.1
  18. 77.5 Gemini 3.8 Flash
  19. 77.4 Claude Opus 4.8
  20. 77.4 Muse Spark 1.2
  21. 77.0 Gemini 3.7 Flash
  22. 76.2 Muse Wonder
  23. 75.9 GPT-5.6 Sol
  24. 75.8 GPT-5.5 Pro
  25. 75.1 Muse Spark 1.3
  26. 75.0 Gemini 4 Argon
  27. 74.7 GPT-5.6 Terra
  28. 74.0 Grok 4.7
  29. 73.8 o3-pro
  30. 73.8 Claude Sonnet 5
  31. 73.4 Claude Sonnet 4.6
  32. 73.3 Qwen 3.8 27B
  33. 73.1 GPT-5
  34. 72.1 GPT-6 Luna
  35. 72.0 Muse Glimmer
  36. 71.6 Grok 4.5
  37. 71.6 Qwen3.8-2.4T-A95B
  38. 71.6 Grok 4.6
  39. 71.5 GPT-5 mini
  40. 71.4 Qwen 3.8 Max
  41. 71.4 o3
  42. 71.4 Kimi K2.6
  43. 71.2 Inkling
  44. 71.2 GPT-5.1
  45. 70.9 DeepSeek-V4-Pro
  46. 70.9 GLM-5.3
  47. 70.8 Qwen 3.6 Max (Preview)
  48. 70.7 Claude Sonnet 4.5
  49. 70.5 GPT-5.4 Mini
  50. 70.5 Kimi K3
  51. 70.2 DeepSeek V4 Pro 0813
  52. 70.0 Gemini 3.6 Flash
  53. 69.6 Grok 4.20 Multi-agent
  54. 68.8 Grok 4.20
  55. 68.6 Gemini 3 Flash
  56. 68.1 DeepSeek V4.1 Flash
  57. 68.0 Gemini 3.5 Flash-Lite
  58. 67.5 gpt-oss-120b
  59. 67.3 GPT-5.4 Nano
  60. 66.7 Claude Haiku 4.5
  61. 66.6 GPT-5.6 Luna
  62. 66.6 Gemini 3.5 Flash
  63. 66.4 Tencent Hy3
  64. 66.2 Qwen Plus
  65. 66.0 GLM-5.2
  66. 66.0 GPT-5.2 Instant
  67. 65.9 MiniMax-M3
  68. 65.8 Claude Opus 4
  69. 65.7 Mistral Large 4
  70. 65.3 Qwen3.7-Max
  71. 65.3 Grok 4.3
  72. 65.0 Gemini 3.1 Pro
  73. 64.6 DeepSeek-V3.2
  74. 64.6 Nemotron 3 Ultra
  75. 64.2 o1
  76. 64.1 Gemini 3.1 Flash-Lite
  77. 63.8 o4-mini
  78. 63.5 Claude Opus 4.1
  79. 62.8 Qwen3-Max
  80. 62.7 Gemini 2.5 Pro
  81. 62.0 Gemma 4 31B IT
  82. 61.9 gpt-oss-20b
  83. 61.8 DeepSeek-V3.2-Exp
  84. 61.7 DeepSeek V4 Flash 0731
  85. 61.5 Qwen 3.6 Flash
  86. 61.1 GPT-5.3 Instant
  87. 60.7 Qwen3.7-Plus
  88. 60.5 Qwen3.5-27B
  89. 59.6 Qwen3-Next-80B-A3B
  90. 59.3 Qwen 3.6 Plus
  91. 58.9 Qwen3.6 27B
  92. 58.8 GPT-5 nano
  93. 58.7 Qwen3 235B-A22B (2507)
  94. 58.2 MiMo-V2.5-Pro
  95. 58.0 Qwen3.5-122B-A10B
  96. 57.3 DeepSeek-V4-Flash
  97. 57.1 Qwen3-235B-A22B-Thinking
  98. 57.0 Qwen3-30B-A3B-Thinking
  99. 57.0 Claude Sonnet 4
  100. 56.5 o3-mini
  101. 56.1 Qwen 3.5 Plus (hosted 397B-A17B)
  102. 56.0 GPT-4.1
  103. 55.6 Qwen3.5 397B-A17B
  104. 55.5 DeepSeek-V3.1
  105. 55.3 Qwen3.5-35B-A3B
  106. 55.2 DeepSeek-V3.1-Terminus
  107. 54.7 Nemotron 3.5 Lightning 30B-A3B
  108. 54.3 Gemma 4 26B A4B
  109. 53.0 Qwen 3.6 35B-A3B
  110. 52.5 Qwen 3.5 Flash (hosted 35B-A3B)
  111. 50.2 Qwen3.5-9B
  112. 50.2 Llama 4 Maverick
  113. 49.7 Llama 4 Scout
  114. 48.7 Mistral Large 3
  115. 48.4 Qwen3-14B
  116. 48.2 Qwen3-32B
  117. 46.8 Qwen3-235B-A22B
  118. 45.6 Mistral Medium 3.5
  119. 45.5 Llama 3.3 70B
  120. 45.4 Qwen3-30B-A3B-Instruct
  121. 45.4 Gemini 2.5 Flash
  122. 45.0 Qwen3-30B-A3B
  123. 44.8 GPT-4o
  124. 43.6 GPT-4.1 mini
  125. 43.4 DeepSeek-V3
  126. 43.2 Llama 3.1-70B
  127. 42.5 Qwen2.5-72B
  128. 42.2 GPT-4 Turbo
  129. 41.8 Qwen3-8B
  130. 41.6 DeepSeek Chat
  131. 41.2 Gemma 3n E4B
  132. 40.5 Gemma 3 27B
  133. 39.2 Mistral Medium 3.1
  134. 39.0 Mistral Small 4
  135. 38.3 Gemma 3 12B
  136. 37.3 GPT-4
  137. 37.3 Cohere Command A
  138. 33.7 Llama 3.1-8B
  139. 33.5 Ministral 3 8B Instruct
  140. 33.4 Gemini 2.5 Flash-Lite
  141. 33.1 Claude 3 Opus
  142. 32.8 Ministral 3 14B
  143. 32.6 GPT-4o mini
  144. 27.3 Qwen2.5-7B
  145. 24.3 GPT-4.1 nano
  146. 23.8 Gemma 3 4B
  147. 20.2 GPT-3.5 Turbo
  148. 18.0 Command R+
  149. 14.7 Claude 3 Haiku
  150. 14.6 Ministral 3B
  151. 13.0 c4ai-command-r-08-2024
  152. 9.94 Command R7B
  1. 1 Claude Opus 5.5 max 86.5
  2. 2 Claude Sonnet 5.5 max 85.1
  3. 3 Claude Fable 5 max 85.0
  4. 4 GPT-6 Sol 82.4
  5. 5 Claude Opus 5 max 82.0
  6. 6 GPT-6 Astra max 81.7
  7. 7 Claude Haiku 5.5 max 81.5
  8. 8 Claude Fable 5.1 max 81.0
  9. 9 GPT-6.1 Sol 80.9
  10. 10 GPT-5.5 80.5
  11. 11 Claude Opus 4.6 80.4
  12. 12 GPT-5.4 80.0
  13. 13 Claude Opus 4.7 79.8
  14. 14 GPT-5.2 79.3
  15. 15 Claude Opus 4.5 79.1
  16. 16 GPT-5.6 Sol Pro 78.9
  17. 17 Muse Spark 1.1 78.4
  18. 18 Gemini 3.8 Flash high 77.5
  19. 19 Claude Opus 4.8 77.4
  20. 20 Muse Spark 1.2 77.4
  21. 21 Gemini 3.7 Flash high 77.0
  22. 22 Muse Wonder 76.2
  23. 23 GPT-5.6 Sol max 75.9
  24. 24 GPT-5.5 Pro 75.8
  25. 25 Muse Spark 1.3 75.1
  26. 26 Gemini 4 Argon high 75.0
  27. 27 GPT-5.6 Terra max 74.7
  28. 28 Grok 4.7 74.0
  29. 29 o3-pro 73.8
  30. 30 Claude Sonnet 5 max 73.8
  31. 31 Claude Sonnet 4.6 73.4
  32. 32 Qwen 3.8 27B 73.3
  33. 33 GPT-5 73.1
  34. 34 GPT-6 Luna 72.1
  35. 35 Muse Glimmer 72.0
  36. 36 Grok 4.5 71.6
  37. 37 Qwen3.8-2.4T-A95B 71.6
  38. 38 Grok 4.6 71.6
  39. 39 GPT-5 mini 71.5
  40. 40 Qwen 3.8 Max 71.4
  41. 41 o3 71.4
  42. 42 Kimi K2.6 71.4
  43. 43 Inkling 71.2
  44. 44 GPT-5.1 71.2
  45. 45 DeepSeek-V4-Pro 70.9
  46. 46 GLM-5.3 70.9
  47. 47 Qwen 3.6 Max (Preview) 70.8
  48. 48 Claude Sonnet 4.5 70.7
  49. 49 GPT-5.4 Mini 70.5
  50. 50 Kimi K3 70.5
  51. 51 DeepSeek V4 Pro 0813 70.2
  52. 52 Gemini 3.6 Flash 70.0
  53. 53 Grok 4.20 Multi-agent 69.6
  54. 54 Grok 4.20 68.8
  55. 55 Gemini 3 Flash 68.6
  56. 56 DeepSeek V4.1 Flash 68.1
  57. 57 Gemini 3.5 Flash-Lite 68.0
  58. 58 gpt-oss-120b 67.5
  59. 59 GPT-5.4 Nano 67.3
  60. 60 Claude Haiku 4.5 66.7
  61. 61 GPT-5.6 Luna max 66.6
  62. 62 Gemini 3.5 Flash 66.6
  63. 63 Tencent Hy3 66.4
  64. 64 Qwen Plus Apr 2025 66.2
  65. 65 GLM-5.2 66.0
  66. 66 GPT-5.2 Instant 66.0
  67. 67 MiniMax-M3 65.9
  68. 68 Claude Opus 4 65.8
  69. 69 Mistral Large 4 65.7
  70. 70 Qwen3.7-Max 65.3
  71. 71 Grok 4.3 65.3
  72. 72 Gemini 3.1 Pro high 65.0
  73. 73 DeepSeek-V3.2 64.6
  74. 74 Nemotron 3 Ultra 64.6
  75. 75 o1 64.2
  76. 76 Gemini 3.1 Flash-Lite 64.1
  77. 77 o4-mini 63.8
  78. 78 Claude Opus 4.1 63.5
  79. 79 Qwen3-Max thinking 62.8
  80. 80 Gemini 2.5 Pro Jun 2025 62.7
  81. 81 Gemma 4 31B IT 62.0
  82. 82 gpt-oss-20b 61.9
  83. 83 DeepSeek-V3.2-Exp 61.8
  84. 84 DeepSeek V4 Flash 0731 61.7
  85. 85 Qwen 3.6 Flash 61.5
  86. 86 GPT-5.3 Instant 61.1
  87. 87 Qwen3.7-Plus 60.7
  88. 88 Qwen3.5-27B 60.5
  89. 89 Qwen3-Next-80B-A3B thinking 59.6
  90. 90 Qwen 3.6 Plus 59.3
  91. 91 Qwen3.6 27B 58.9
  92. 92 GPT-5 nano 58.8
  93. 93 Qwen3 235B-A22B (2507) 58.7
  94. 94 MiMo-V2.5-Pro 58.2
  95. 95 Qwen3.5-122B-A10B 58.0
  96. 96 DeepSeek-V4-Flash 57.3
  97. 97 Qwen3-235B-A22B-Thinking Jul 2025 57.1
  98. 98 Qwen3-30B-A3B-Thinking Jul 2025 57.0
  99. 99 Claude Sonnet 4 57.0
  100. 100 o3-mini 56.5
  101. 101 Qwen 3.5 Plus (hosted 397B-A17B) 56.1
  102. 102 GPT-4.1 56.0
  103. 103 Qwen3.5 397B-A17B 55.6
  104. 104 DeepSeek-V3.1 non-thinking 55.5
  105. 105 Qwen3.5-35B-A3B 55.3
  106. 106 DeepSeek-V3.1-Terminus 55.2
  107. 107 Nemotron 3.5 Lightning 30B-A3B 54.7
  108. 108 Gemma 4 26B A4B 54.3
  109. 109 Qwen 3.6 35B-A3B 53.0
  110. 110 Qwen 3.5 Flash (hosted 35B-A3B) 52.5
  111. 111 Qwen3.5-9B 50.2
  112. 112 Llama 4 Maverick 50.2
  113. 113 Llama 4 Scout 49.7
  114. 114 Mistral Large 3 48.7
  115. 115 Qwen3-14B 48.4
  116. 116 Qwen3-32B 48.2
  117. 117 Qwen3-235B-A22B 46.8
  118. 118 Mistral Medium 3.5 45.6
  119. 119 Llama 3.3 70B 45.5
  120. 120 Qwen3-30B-A3B-Instruct Jul 2025 45.4
  121. 121 Gemini 2.5 Flash Jun 2025 45.4
  122. 122 Qwen3-30B-A3B 45.0
  123. 123 GPT-4o May 2024 44.8
  124. 124 GPT-4.1 mini 43.6
  125. 125 DeepSeek-V3 Mar 2025 43.4
  126. 126 Llama 3.1-70B 43.2
  127. 127 Qwen2.5-72B 42.5
  128. 128 GPT-4 Turbo Apr 2024 42.2
  129. 129 Qwen3-8B 41.8
  130. 130 DeepSeek Chat 41.6
  131. 131 Gemma 3n E4B 41.2
  132. 132 Gemma 3 27B 40.5
  133. 133 Mistral Medium 3.1 39.2
  134. 134 Mistral Small 4 39.0
  135. 135 Gemma 3 12B 38.3
  136. 136 GPT-4 Jun 2023 37.3
  137. 137 Cohere Command A 37.3
  138. 138 Llama 3.1-8B 33.7
  139. 139 Ministral 3 8B Instruct 33.5
  140. 140 Gemini 2.5 Flash-Lite Jun 2025 33.4
  141. 141 Claude 3 Opus 33.1
  142. 142 Ministral 3 14B 32.8
  143. 143 GPT-4o mini 32.6
  144. 144 Qwen2.5-7B 27.3
  145. 145 GPT-4.1 nano 24.3
  146. 146 Gemma 3 4B 23.8
  147. 147 GPT-3.5 Turbo Nov 2023 20.2
  148. 148 Command R+ 18.0
  149. 149 Claude 3 Haiku 14.7
  150. 150 Ministral 3B 14.6
  151. 151 c4ai-command-r-08-2024 13.0
  152. 152 Command R7B 9.94
#EntryScoreSettingDateSource
1 Claude Opus 5.5 86.5 max Sep 2026 Official leaderboard
2 Claude Sonnet 5.5 85.1 max Sep 2026 Official leaderboard
3 Claude Fable 5 85.0 max Jun 2026 Official leaderboard
4 GPT-6 Sol 82.4 Sep 2026 Official leaderboard
5 Claude Opus 5 82.0 max Jul 2026 Official leaderboard
6 GPT-6 Astra 81.7 max Sep 2026 Official leaderboard
7 Claude Haiku 5.5 81.5 max Oct 2026 Official leaderboard
8 Claude Fable 5.1 81.0 max Sep 2026 Official leaderboard
9 GPT-6.1 Sol 80.9 Sep 2026 Official leaderboard
10 GPT-5.5 80.5 Apr 2026 Official leaderboard
11 Claude Opus 4.6 80.4 Feb 2026 Official leaderboard
12 GPT-5.4 80.0 Mar 2026 Official leaderboard
13 Claude Opus 4.7 79.8 Apr 2026 Official leaderboard
14 GPT-5.2 79.3 Dec 2025 Official leaderboard
15 Claude Opus 4.5 79.1 Nov 2025 Official leaderboard
16 GPT-5.6 Sol Pro 78.9 Jul 2026 Official leaderboard
17 Muse Spark 1.1 78.4 Jul 2026 Official leaderboard
18 Gemini 3.8 Flash 77.5 high Sep 2026 Official leaderboard
19 Claude Opus 4.8 77.4 May 2026 Official leaderboard
20 Muse Spark 1.2 77.4 Aug 2026 Official leaderboard
21 Gemini 3.7 Flash 77.0 high Aug 2026 Official leaderboard
22 Muse Wonder 76.2 Oct 2026 Official leaderboard
23 GPT-5.6 Sol 75.9 max Jul 2026 Official leaderboard
24 GPT-5.5 Pro 75.8 Apr 2026 Official leaderboard
25 Muse Spark 1.3 75.1 Sep 2026 Official leaderboard
26 Gemini 4 Argon 75.0 high Sep 2026 Official leaderboard
27 GPT-5.6 Terra 74.7 max Jul 2026 Official leaderboard
28 Grok 4.7 74.0 Sep 2026 Official leaderboard
29 o3-pro 73.8 Jun 2025 Official leaderboard
30 Claude Sonnet 5 73.8 max Jun 2026 Official leaderboard
31 Claude Sonnet 4.6 73.4 Feb 2026 Official leaderboard
32 Qwen 3.8 27B 73.3 Aug 2026 Official leaderboard
33 GPT-5 73.1 Aug 2025 Official leaderboard
34 GPT-6 Luna 72.1 Sep 2026 Official leaderboard
35 Muse Glimmer 72.0 Aug 2026 Official leaderboard
36 Grok 4.5 71.6 Jul 2026 Official leaderboard
37 Qwen3.8-2.4T-A95B 71.6 Aug 2026 Official leaderboard
38 Grok 4.6 71.6 Aug 2026 Official leaderboard
39 GPT-5 mini 71.5 Aug 2025 Official leaderboard
40 Qwen 3.8 Max 71.4 Aug 2026 Official leaderboard
41 o3 71.4 Apr 2025 Official leaderboard
42 Kimi K2.6 71.4 Apr 2026 Official leaderboard
43 Inkling 71.2 Jul 2026 Official leaderboard
44 GPT-5.1 71.2 Nov 2025 Official leaderboard
45 DeepSeek-V4-Pro 70.9 Apr 2026 Official leaderboard
46 GLM-5.3 70.9 Aug 2026 Official leaderboard
47 Qwen 3.6 Max (Preview) 70.8 Apr 2026 Official leaderboard
48 Claude Sonnet 4.5 70.7 Sep 2025 Official leaderboard
49 GPT-5.4 Mini 70.5 Mar 2026 Official leaderboard
50 Kimi K3 70.5 Jul 2026 Official leaderboard
51 DeepSeek V4 Pro 0813 70.2 Aug 2026 Official leaderboard
52 Gemini 3.6 Flash 70.0 Jul 2026 Official leaderboard
53 Grok 4.20 Multi-agent 69.6 Mar 2026 Official leaderboard
54 Grok 4.20 68.8 Feb 2026 Official leaderboard
55 Gemini 3 Flash 68.6 Dec 2025 Official leaderboard
56 DeepSeek V4.1 Flash 68.1 Sep 2026 Official leaderboard
57 Gemini 3.5 Flash-Lite 68.0 Jul 2026 Official leaderboard
58 gpt-oss-120b 67.5 Aug 2025 Official leaderboard
59 GPT-5.4 Nano 67.3 Mar 2026 Official leaderboard
60 Claude Haiku 4.5 66.7 Oct 2025 Official leaderboard
61 GPT-5.6 Luna 66.6 max Jul 2026 Official leaderboard
62 Gemini 3.5 Flash 66.6 May 2026 Official leaderboard
63 Tencent Hy3 66.4 Jul 2026 Official leaderboard
64 Qwen Plus Apr 2025 66.2 Apr 2025 Official leaderboard
65 GLM-5.2 66.0 Jun 2026 Official leaderboard
66 GPT-5.2 Instant 66.0 Dec 2025 Official leaderboard
67 MiniMax-M3 65.9 Jun 2026 Official leaderboard
68 Claude Opus 4 65.8 May 2025 Official leaderboard
69 Mistral Large 4 65.7 Oct 2026 Official leaderboard
70 Qwen3.7-Max 65.3 May 2026 Official leaderboard
71 Grok 4.3 65.3 Apr 2026 Official leaderboard
72 Gemini 3.1 Pro 65.0 high Feb 2026 Official leaderboard
73 DeepSeek-V3.2 64.6 Dec 2025 Official leaderboard
74 Nemotron 3 Ultra 64.6 Jun 2026 Official leaderboard
75 o1 64.2 Dec 2024 Official leaderboard
76 Gemini 3.1 Flash-Lite 64.1 Mar 2026 Official leaderboard
77 o4-mini 63.8 Apr 2025 Official leaderboard
78 Claude Opus 4.1 63.5 Aug 2025 Official leaderboard
79 Qwen3-Max 62.8 thinking Sep 2025 Official leaderboard
80 Gemini 2.5 Pro Jun 2025 62.7 Jun 2025 Official leaderboard
81 Gemma 4 31B IT 62.0 Apr 2026 Official leaderboard
82 gpt-oss-20b 61.9 Aug 2025 Official leaderboard
83 DeepSeek-V3.2-Exp 61.8 Sep 2025 Official leaderboard
84 DeepSeek V4 Flash 0731 61.7 Jul 2026 Official leaderboard
85 Qwen 3.6 Flash 61.5 Apr 2026 Official leaderboard
86 GPT-5.3 Instant 61.1 Mar 2026 Official leaderboard
87 Qwen3.7-Plus 60.7 Jun 2026 Official leaderboard
88 Qwen3.5-27B 60.5 Feb 2026 Official leaderboard
89 Qwen3-Next-80B-A3B 59.6 thinking Sep 2025 Official leaderboard
90 Qwen 3.6 Plus 59.3 Mar 2026 Official leaderboard
91 Qwen3.6 27B 58.9 Apr 2026 Official leaderboard
92 GPT-5 nano 58.8 Aug 2025 Official leaderboard
93 Qwen3 235B-A22B (2507) 58.7 Jul 2025 Official leaderboard
94 MiMo-V2.5-Pro 58.2 Apr 2026 Official leaderboard
95 Qwen3.5-122B-A10B 58.0 Feb 2026 Official leaderboard
96 DeepSeek-V4-Flash 57.3 Apr 2026 Official leaderboard
97 Qwen3-235B-A22B-Thinking Jul 2025 57.1 Jul 2025 Official leaderboard
98 Qwen3-30B-A3B-Thinking Jul 2025 57.0 Jul 2025 Official leaderboard
99 Claude Sonnet 4 57.0 May 2025 Official leaderboard
100 o3-mini 56.5 Jan 2025 Official leaderboard
101 Qwen 3.5 Plus (hosted 397B-A17B) 56.1 Feb 2026 Official leaderboard
102 GPT-4.1 56.0 Apr 2025 Official leaderboard
103 Qwen3.5 397B-A17B 55.6 Feb 2026 Official leaderboard
104 DeepSeek-V3.1 55.5 non-thinking Aug 2025 Official leaderboard
105 Qwen3.5-35B-A3B 55.3 Feb 2026 Official leaderboard
106 DeepSeek-V3.1-Terminus 55.2 Sep 2025 Official leaderboard
107 Nemotron 3.5 Lightning 30B-A3B 54.7 Aug 2026 Official leaderboard
108 Gemma 4 26B A4B 54.3 Apr 2026 Official leaderboard
109 Qwen 3.6 35B-A3B 53.0 Apr 2026 Official leaderboard
110 Qwen 3.5 Flash (hosted 35B-A3B) 52.5 Feb 2026 Official leaderboard
111 Qwen3.5-9B 50.2 Feb 2026 Official leaderboard
112 Llama 4 Maverick 50.2 Apr 2025 Official leaderboard
113 Llama 4 Scout 49.7 Apr 2025 Official leaderboard
114 Mistral Large 3 48.7 Dec 2025 Official leaderboard
115 Qwen3-14B 48.4 Apr 2025 Official leaderboard
116 Qwen3-32B 48.2 Apr 2025 Official leaderboard
117 Qwen3-235B-A22B 46.8 Apr 2025 Official leaderboard
118 Mistral Medium 3.5 45.6 Apr 2026 Official leaderboard
119 Llama 3.3 70B 45.5 Dec 2024 Official leaderboard
120 Qwen3-30B-A3B-Instruct Jul 2025 45.4 Jul 2025 Official leaderboard
121 Gemini 2.5 Flash Jun 2025 45.4 Jun 2025 Official leaderboard
122 Qwen3-30B-A3B 45.0 Apr 2025 Official leaderboard
123 GPT-4o May 2024 44.8 May 2024 Official leaderboard
124 GPT-4.1 mini 43.6 Apr 2025 Official leaderboard
125 DeepSeek-V3 Mar 2025 43.4 Mar 2025 Official leaderboard
126 Llama 3.1-70B 43.2 Jul 2024 Official leaderboard
127 Qwen2.5-72B 42.5 Sep 2024 Official leaderboard
128 GPT-4 Turbo Apr 2024 42.2 Apr 2024 Official leaderboard
129 Qwen3-8B 41.8 Apr 2025 Official leaderboard
130 DeepSeek Chat 41.6 Dec 2024 Official leaderboard
131 Gemma 3n E4B 41.2 Jun 2025 Official leaderboard
132 Gemma 3 27B 40.5 Mar 2025 Official leaderboard
133 Mistral Medium 3.1 39.2 Aug 2025 Official leaderboard
134 Mistral Small 4 39.0 Mar 2026 Official leaderboard
135 Gemma 3 12B 38.3 Mar 2025 Official leaderboard
136 GPT-4 Jun 2023 37.3 Jun 2023 Official leaderboard
137 Cohere Command A 37.3 Mar 2025 Official leaderboard
138 Llama 3.1-8B 33.7 Jul 2024 Official leaderboard
139 Ministral 3 8B Instruct 33.5 Dec 2025 Official leaderboard
140 Gemini 2.5 Flash-Lite Jun 2025 33.4 Jun 2025 Official leaderboard
141 Claude 3 Opus 33.1 Feb 2024 Official leaderboard
142 Ministral 3 14B 32.8 Dec 2025 Official leaderboard
143 GPT-4o mini 32.6 Jul 2024 Official leaderboard
144 Qwen2.5-7B 27.3 Sep 2024 Official leaderboard
145 GPT-4.1 nano 24.3 Apr 2025 Official leaderboard
146 Gemma 3 4B 23.8 Mar 2025 Official leaderboard
147 GPT-3.5 Turbo Nov 2023 20.2 Nov 2023 Official leaderboard
148 Command R+ 18.0 Aug 2024 Official leaderboard
149 Claude 3 Haiku 14.7 Mar 2024 Official leaderboard
150 Ministral 3B 14.6 Oct 2024 Official leaderboard
151 c4ai-command-r-08-2024 13.0 Aug 2024 Official leaderboard
152 Command R7B 9.94 Dec 2024 Official leaderboard

2 models have held number one since its release in Aug 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 2 leaders
Claude Opus 5.5 86.5

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0.00 20.0 40.0 60.0 80.0 100.0 2023 2024 2025 2026 0.00 20.0 40.0 60.0 80.0 100.0 2023 2024 2025 2026

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Other

All 41 in Other
  • Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
  • Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Gemini 4 Carbon 59.0
  • Anthropic ECI (AECI) Only in system cards Index that combines a basket of capability benchmarks into one capability score to track trends across Anthropic models. Claude Opus 5.5 169.4
  • Qwen in-house English automatic evaluation Only in system cards Models answer Qwen in-house English datasets on knowledge, comprehension, coding, math and related skills. Skills and weights change between reports. Qwen2-72B 72.9%
  • Recognition of model-written user turns (Anthropic) Only in system cards Models write synthetic user turns for real Claude Code transcripts, then judge which version is real. Score is accuracy above chance. Claude Mythos Preview 0.35
  • LiveBench Dropped Models answer monthly refreshed questions across math, coding, reasoning, language, instruction following, and data analysis. Scored automatically against objective ground truth. Claude Fable 5.1 85.7%