Benchmark Almanac

Conceptual Reasoning Index

Claude Opus 5.5 77.9 leads since Sep 2026 Not in model cards

An index that ranks models on reasoning about hard-to-verify topics such as AI alignment and decision theory. It weights LMCA 60 percent and ACCoRD and DTBench 20 percent each.

The leaderboard CSV had 267 model rows in November 2026.

Made by Redwood Research (conceptual reasoning team) Released Aug 2026 Composite indices All tasks public Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
Leaderboard conceptualreasoning.ai
152
Entries on its own leaderboard, 149 models we track

Claude Opus 5.5 leads with 77.9

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

152 entries. Scroll sideways for the rest.

  1. 77.9 Claude Opus 5.5
  2. 77.1 Claude Sonnet 5.5
  3. 74.7 Claude Fable 5.1
  4. 73.8 GPT-6 Astra
  5. 73.6 Claude Opus 5
  6. 72.7 GPT-6.1 Sol
  7. 72.7 Claude Fable 5
  8. 72.3 Gemini 4 Argon
  9. 71.0 GPT-6 Sol
  10. 70.0 GPT-5.6 Sol Pro
  11. 68.7 GPT-5.6 Sol
  12. 68.3 Claude Opus 4.8
  13. 67.7 Muse Wonder
  14. 67.3 GPT-5.5
  15. 66.6 Claude Opus 4.6
  16. 66.2 GPT-5.5 Pro
  17. 66.2 Muse Spark 1.3
  18. 65.7 GPT-5.6 Terra
  19. 65.5 Claude Opus 4.7
  20. 65.3 GPT-5.4
  21. 65.3 Claude Haiku 5.5
  22. 65.0 Gemini 3.8 Flash
  23. 64.6 Gemini 3.7 Flash
  24. 63.8 Gemini 3.1 Pro
  25. 63.8 Muse Spark 1.1
  26. 63.4 GLM-5.3
  27. 63.1 Grok 4.7
  28. 62.8 Kimi K3
  29. 62.7 Muse Spark 1.2
  30. 62.5 Grok 4.6
  31. 61.8 Claude Sonnet 5
  32. 60.3 Grok 4.5
  33. 59.8 Gemini 3.5 Flash
  34. 59.5 Gemini 3.6 Flash
  35. 59.4 Qwen 3.8 Max
  36. 59.4 Qwen3.8-2.4T-A95B
  37. 59.3 DeepSeek V4 Pro 0813
  38. 59.2 Claude Sonnet 4.6
  39. 59.2 GPT-5.2
  40. 59.1 Claude Opus 4.5
  41. 58.7 GPT-5.6 Luna
  42. 58.6 GLM-5.2
  43. 58.4 DeepSeek V4.1 Flash
  44. 57.8 GPT-6 Luna
  45. 57.3 GPT-5.1
  46. 56.9 Qwen3.7-Max
  47. 56.0 Gemini 3 Flash
  48. 55.8 DeepSeek-V4-Pro
  49. 55.6 Grok 4.20 Multi-agent
  50. 55.5 GPT-5
  51. 55.5 Qwen 3.8 27B
  52. 55.4 Qwen 3.6 Max (Preview)
  53. 54.4 DeepSeek V4 Flash 0731
  54. 53.7 Grok 4.20
  55. 53.6 Kimi K2.6
  56. 53.5 o3-pro
  57. 53.0 o3
  58. 52.9 Grok 4.3
  59. 52.6 Inkling
  60. 52.2 Tencent Hy3
  61. 51.9 GPT-5.4 Mini
  62. 51.8 Claude Sonnet 4.5
  63. 51.8 Nemotron 3 Ultra
  64. 51.0 GPT-5.2 Instant
  65. 50.7 Mistral Large 4
  66. 50.7 Gemini 3.5 Flash-Lite
  67. 50.5 Muse Glimmer 30B
  68. 50.2 Gemma 4 31B IT
  69. 49.7 Qwen3.5 397B-A17B
  70. 49.5 Claude Opus 4
  71. 49.4 Qwen3.7-Plus
  72. 49.0 GPT-5.4 Nano
  73. 48.6 Qwen3-Max
  74. 48.5 DeepSeek-V4-Flash
  75. 48.3 GPT-5 mini
  76. 48.3 Claude Opus 4.1
  77. 48.1 GPT-5.3 Instant
  78. 47.5 Gemini 2.5 Pro
  79. 46.6 Qwen3.5-27B
  80. 46.5 Qwen 3.5 Plus (hosted 397B-A17B)
  81. 46.4 MiniMax-M3
  82. 46.1 Gemini 3.1 Flash-Lite
  83. 45.7 DeepSeek-V3.2-Exp
  84. 45.7 Qwen3.5-122B-A10B
  85. 45.7 Qwen 3.6 Plus
  86. 45.4 DeepSeek-V3.2
  87. 45.2 Qwen3.6 27B
  88. 44.2 MiMo-V2.5-Pro
  89. 43.3 Qwen 3.6 Flash
  90. 43.1 Claude Haiku 4.5
  91. 42.4 Qwen3-235B-A22B-Thinking
  92. 42.3 Qwen 3.5 Flash (hosted 35B-A3B)
  93. 42.1 Qwen3.5-35B-A3B
  94. 42.0 DeepSeek-V3.1-Terminus
  95. 41.3 Qwen Plus
  96. 41.2 o4-mini
  97. 41.1 Claude Sonnet 4
  98. 40.3 Gemma 4 26B A4B
  99. 39.9 DeepSeek-V3.1
  100. 39.7 Qwen 3.6 35B-A3B
  101. 38.9 gpt-oss-120b
  102. 38.7 Qwen3 235B-A22B (2507)
  103. 37.8 o1
  104. 37.7 Gemini 2.5 Flash
  105. 36.7 Qwen3-Next-80B-A3B
  106. 36.6 Mistral Medium 3.5
  107. 36.3 Qwen3-235B-A22B
  108. 36.0 GPT-4.1
  109. 35.1 Qwen3.5-9B
  110. 33.6 Nemotron 3.5 Lightning 30B-A3B
  111. 32.9 Qwen3-30B-A3B-Thinking
  112. 32.3 o3-mini
  113. 31.6 Qwen3-30B-A3B-Instruct
  114. 31.0 GPT-4.1 mini
  115. 30.5 Mistral Small 4
  116. 30.4 gpt-oss-20b
  117. 29.2 Qwen3-32B
  118. 28.6 Qwen3-14B
  119. 28.4 Mistral Medium 3.1
  120. 28.1 Mistral Large 3
  121. 27.1 GPT-4o
  122. 26.8 Llama 4 Maverick
  123. 26.2 DeepSeek-V3
  124. 26.1 Llama 3.3 70B
  125. 25.3 GPT-4
  126. 25.2 Qwen3-30B-A3B
  127. 25.1 Gemini 2.5 Flash-Lite
  128. 25.0 DeepSeek Chat
  129. 24.2 Llama 3.1-70B
  130. 24.2 Qwen2.5-72B
  131. 24.1 GPT-5 nano
  132. 24.0 Claude 3 Opus
  133. 23.1 Llama 4 Scout
  134. 21.5 GPT-4 Turbo
  135. 20.7 Cohere Command A
  136. 20.2 Qwen3-8B
  137. 19.6 Gemma 3 27B
  138. 19.3 Ministral 3 14B
  139. 17.5 GPT-4o mini
  140. 17.4 Ministral 3 8B Instruct
  141. 13.6 Llama 3.1-8B
  142. 13.3 Gemma 3n E4B
  143. 13.3 Gemma 3 12B
  144. 12.7 GPT-3.5 Turbo
  145. 12.3 GPT-4.1 nano
  146. 11.9 Qwen2.5-7B
  147. 11.6 Claude 3 Haiku
  148. 11.6 Command R+
  149. 10.3 c4ai-command-r-08-2024
  150. 10.1 Ministral 3B
  151. 10.1 Gemma 3 4B
  152. 7.1 Command R7B
  1. 1 Claude Opus 5.5 max 77.9
  2. 2 Claude Sonnet 5.5 max 77.1
  3. 3 Claude Fable 5.1 max 74.7
  4. 4 GPT-6 Astra max 73.8
  5. 5 Claude Opus 5 max 73.6
  6. 6 GPT-6.1 Sol 72.7
  7. 7 Claude Fable 5 max 72.7
  8. 8 Gemini 4 Argon high 72.3
  9. 9 GPT-6 Sol 71.0
  10. 10 GPT-5.6 Sol Pro 70.0
  11. 11 GPT-5.6 Sol max 68.7
  12. 12 Claude Opus 4.8 68.3
  13. 13 Muse Wonder 67.7
  14. 14 GPT-5.5 67.3
  15. 15 Claude Opus 4.6 66.6
  16. 16 GPT-5.5 Pro 66.2
  17. 17 Muse Spark 1.3 66.2
  18. 18 GPT-5.6 Terra max 65.7
  19. 19 Claude Opus 4.7 65.5
  20. 20 GPT-5.4 65.3
  21. 21 Claude Haiku 5.5 max 65.3
  22. 22 Gemini 3.8 Flash high 65.0
  23. 23 Gemini 3.7 Flash high 64.6
  24. 24 Gemini 3.1 Pro high 63.8
  25. 25 Muse Spark 1.1 63.8
  26. 26 GLM-5.3 63.4
  27. 27 Grok 4.7 63.1
  28. 28 Kimi K3 62.8
  29. 29 Muse Spark 1.2 62.7
  30. 30 Grok 4.6 62.5
  31. 31 Claude Sonnet 5 max 61.8
  32. 32 Grok 4.5 60.3
  33. 33 Gemini 3.5 Flash 59.8
  34. 34 Gemini 3.6 Flash 59.5
  35. 35 Qwen 3.8 Max 59.4
  36. 36 Qwen3.8-2.4T-A95B 59.4
  37. 37 DeepSeek V4 Pro 0813 59.3
  38. 38 Claude Sonnet 4.6 59.2
  39. 39 GPT-5.2 59.2
  40. 40 Claude Opus 4.5 59.1
  41. 41 GPT-5.6 Luna max 58.7
  42. 42 GLM-5.2 58.6
  43. 43 DeepSeek V4.1 Flash 58.4
  44. 44 GPT-6 Luna 57.8
  45. 45 GPT-5.1 57.3
  46. 46 Qwen3.7-Max 56.9
  47. 47 Gemini 3 Flash 56.0
  48. 48 DeepSeek-V4-Pro 55.8
  49. 49 Grok 4.20 Multi-agent 55.6
  50. 50 GPT-5 55.5
  51. 51 Qwen 3.8 27B 55.5
  52. 52 Qwen 3.6 Max (Preview) 55.4
  53. 53 DeepSeek V4 Flash 0731 54.4
  54. 54 Grok 4.20 53.7
  55. 55 Kimi K2.6 53.6
  56. 56 o3-pro 53.5
  57. 57 o3 53.0
  58. 58 Grok 4.3 52.9
  59. 59 Inkling 52.6
  60. 60 Tencent Hy3 52.2
  61. 61 GPT-5.4 Mini 51.9
  62. 62 Claude Sonnet 4.5 51.8
  63. 63 Nemotron 3 Ultra 51.8
  64. 64 GPT-5.2 Instant 51.0
  65. 65 Mistral Large 4 50.7
  66. 66 Gemini 3.5 Flash-Lite 50.7
  67. 67 Muse Glimmer 30B 50.5
  68. 68 Gemma 4 31B IT 50.2
  69. 69 Qwen3.5 397B-A17B 49.7
  70. 70 Claude Opus 4 49.5
  71. 71 Qwen3.7-Plus 49.4
  72. 72 GPT-5.4 Nano 49.0
  73. 73 Qwen3-Max thinking 48.6
  74. 74 DeepSeek-V4-Flash 48.5
  75. 75 GPT-5 mini 48.3
  76. 76 Claude Opus 4.1 48.3
  77. 77 GPT-5.3 Instant 48.1
  78. 78 Gemini 2.5 Pro Jun 2025 47.5
  79. 79 Qwen3.5-27B 46.6
  80. 80 Qwen 3.5 Plus (hosted 397B-A17B) 46.5
  81. 81 MiniMax-M3 46.4
  82. 82 Gemini 3.1 Flash-Lite 46.1
  83. 83 DeepSeek-V3.2-Exp 45.7
  84. 84 Qwen3.5-122B-A10B 45.7
  85. 85 Qwen 3.6 Plus 45.7
  86. 86 DeepSeek-V3.2 45.4
  87. 87 Qwen3.6 27B 45.2
  88. 88 MiMo-V2.5-Pro 44.2
  89. 89 Qwen 3.6 Flash 43.3
  90. 90 Claude Haiku 4.5 43.1
  91. 91 Qwen3-235B-A22B-Thinking Jul 2025 42.4
  92. 92 Qwen 3.5 Flash (hosted 35B-A3B) 42.3
  93. 93 Qwen3.5-35B-A3B 42.1
  94. 94 DeepSeek-V3.1-Terminus 42.0
  95. 95 Qwen Plus Apr 2025 41.3
  96. 96 o4-mini 41.2
  97. 97 Claude Sonnet 4 41.1
  98. 98 Gemma 4 26B A4B 40.3
  99. 99 DeepSeek-V3.1 non-thinking 39.9
  100. 100 Qwen 3.6 35B-A3B 39.7
  101. 101 gpt-oss-120b 38.9
  102. 102 Qwen3 235B-A22B (2507) 38.7
  103. 103 o1 37.8
  104. 104 Gemini 2.5 Flash Jun 2025 37.7
  105. 105 Qwen3-Next-80B-A3B thinking 36.7
  106. 106 Mistral Medium 3.5 36.6
  107. 107 Qwen3-235B-A22B 36.3
  108. 108 GPT-4.1 36.0
  109. 109 Qwen3.5-9B 35.1
  110. 110 Nemotron 3.5 Lightning 30B-A3B 33.6
  111. 111 Qwen3-30B-A3B-Thinking Jul 2025 32.9
  112. 112 o3-mini 32.3
  113. 113 Qwen3-30B-A3B-Instruct Jul 2025 31.6
  114. 114 GPT-4.1 mini 31.0
  115. 115 Mistral Small 4 30.5
  116. 116 gpt-oss-20b 30.4
  117. 117 Qwen3-32B 29.2
  118. 118 Qwen3-14B 28.6
  119. 119 Mistral Medium 3.1 28.4
  120. 120 Mistral Large 3 28.1
  121. 121 GPT-4o May 2024 27.1
  122. 122 Llama 4 Maverick 26.8
  123. 123 DeepSeek-V3 Mar 2025 26.2
  124. 124 Llama 3.3 70B 26.1
  125. 125 GPT-4 Jun 2023 25.3
  126. 126 Qwen3-30B-A3B 25.2
  127. 127 Gemini 2.5 Flash-Lite Jun 2025 25.1
  128. 128 DeepSeek Chat 25.0
  129. 129 Llama 3.1-70B 24.2
  130. 130 Qwen2.5-72B 24.2
  131. 131 GPT-5 nano 24.1
  132. 132 Claude 3 Opus 24.0
  133. 133 Llama 4 Scout 23.1
  134. 134 GPT-4 Turbo Apr 2024 21.5
  135. 135 Cohere Command A 20.7
  136. 136 Qwen3-8B 20.2
  137. 137 Gemma 3 27B 19.6
  138. 138 Ministral 3 14B 19.3
  139. 139 GPT-4o mini 17.5
  140. 140 Ministral 3 8B Instruct 17.4
  141. 141 Llama 3.1-8B 13.6
  142. 142 Gemma 3n E4B 13.3
  143. 143 Gemma 3 12B 13.3
  144. 144 GPT-3.5 Turbo Nov 2023 12.7
  145. 145 GPT-4.1 nano 12.3
  146. 146 Qwen2.5-7B 11.9
  147. 147 Claude 3 Haiku 11.6
  148. 148 Command R+ 11.6
  149. 149 c4ai-command-r-08-2024 10.3
  150. 150 Ministral 3B 10.1
  151. 151 Gemma 3 4B 10.1
  152. 152 Command R7B 7.1
#EntryScoreSettingDateSource
1 Claude Opus 5.5 77.9 max Sep 2026 Official leaderboard
2 Claude Sonnet 5.5 77.1 max Sep 2026 Official leaderboard
3 Claude Fable 5.1 74.7 max Sep 2026 Official leaderboard
4 GPT-6 Astra 73.8 max Sep 2026 Official leaderboard
5 Claude Opus 5 73.6 max Jul 2026 Official leaderboard
6 GPT-6.1 Sol 72.7 Sep 2026 Official leaderboard
7 Claude Fable 5 72.7 max Jun 2026 Official leaderboard
8 Gemini 4 Argon 72.3 high Sep 2026 Official leaderboard
9 GPT-6 Sol 71.0 Sep 2026 Official leaderboard
10 GPT-5.6 Sol Pro 70.0 Jul 2026 Official leaderboard
11 GPT-5.6 Sol 68.7 max Jul 2026 Official leaderboard
12 Claude Opus 4.8 68.3 May 2026 Official leaderboard
13 Muse Wonder 67.7 Oct 2026 Official leaderboard
14 GPT-5.5 67.3 Apr 2026 Official leaderboard
15 Claude Opus 4.6 66.6 Feb 2026 Official leaderboard
16 GPT-5.5 Pro 66.2 Apr 2026 Official leaderboard
17 Muse Spark 1.3 66.2 Sep 2026 Official leaderboard
18 GPT-5.6 Terra 65.7 max Jul 2026 Official leaderboard
19 Claude Opus 4.7 65.5 Apr 2026 Official leaderboard
20 GPT-5.4 65.3 Mar 2026 Official leaderboard
21 Claude Haiku 5.5 65.3 max Oct 2026 Official leaderboard
22 Gemini 3.8 Flash 65.0 high Sep 2026 Official leaderboard
23 Gemini 3.7 Flash 64.6 high Aug 2026 Official leaderboard
24 Gemini 3.1 Pro 63.8 high Feb 2026 Official leaderboard
25 Muse Spark 1.1 63.8 Jul 2026 Official leaderboard
26 GLM-5.3 63.4 Aug 2026 Official leaderboard
27 Grok 4.7 63.1 Sep 2026 Official leaderboard
28 Kimi K3 62.8 Jul 2026 Official leaderboard
29 Muse Spark 1.2 62.7 Aug 2026 Official leaderboard
30 Grok 4.6 62.5 Aug 2026 Official leaderboard
31 Claude Sonnet 5 61.8 max Jun 2026 Official leaderboard
32 Grok 4.5 60.3 Jul 2026 Official leaderboard
33 Gemini 3.5 Flash 59.8 May 2026 Official leaderboard
34 Gemini 3.6 Flash 59.5 Jul 2026 Official leaderboard
35 Qwen 3.8 Max 59.4 Aug 2026 Official leaderboard
36 Qwen3.8-2.4T-A95B 59.4 Aug 2026 Official leaderboard
37 DeepSeek V4 Pro 0813 59.3 Aug 2026 Official leaderboard
38 Claude Sonnet 4.6 59.2 Feb 2026 Official leaderboard
39 GPT-5.2 59.2 Dec 2025 Official leaderboard
40 Claude Opus 4.5 59.1 Nov 2025 Official leaderboard
41 GPT-5.6 Luna 58.7 max Jul 2026 Official leaderboard
42 GLM-5.2 58.6 Jun 2026 Official leaderboard
43 DeepSeek V4.1 Flash 58.4 Sep 2026 Official leaderboard
44 GPT-6 Luna 57.8 Sep 2026 Official leaderboard
45 GPT-5.1 57.3 Nov 2025 Official leaderboard
46 Qwen3.7-Max 56.9 May 2026 Official leaderboard
47 Gemini 3 Flash 56.0 Dec 2025 Official leaderboard
48 DeepSeek-V4-Pro 55.8 Apr 2026 Official leaderboard
49 Grok 4.20 Multi-agent 55.6 Mar 2026 Official leaderboard
50 GPT-5 55.5 Aug 2025 Official leaderboard
51 Qwen 3.8 27B 55.5 Aug 2026 Official leaderboard
52 Qwen 3.6 Max (Preview) 55.4 Apr 2026 Official leaderboard
53 DeepSeek V4 Flash 0731 54.4 Jul 2026 Official leaderboard
54 Grok 4.20 53.7 Feb 2026 Official leaderboard
55 Kimi K2.6 53.6 Apr 2026 Official leaderboard
56 o3-pro 53.5 Jun 2025 Official leaderboard
57 o3 53.0 Apr 2025 Official leaderboard
58 Grok 4.3 52.9 Apr 2026 Official leaderboard
59 Inkling 52.6 Jul 2026 Official leaderboard
60 Tencent Hy3 52.2 Jul 2026 Official leaderboard
61 GPT-5.4 Mini 51.9 Mar 2026 Official leaderboard
62 Claude Sonnet 4.5 51.8 Sep 2025 Official leaderboard
63 Nemotron 3 Ultra 51.8 Jun 2026 Official leaderboard
64 GPT-5.2 Instant 51.0 Dec 2025 Official leaderboard
65 Mistral Large 4 50.7 Oct 2026 Official leaderboard
66 Gemini 3.5 Flash-Lite 50.7 Jul 2026 Official leaderboard
67 Muse Glimmer 30B 50.5 Aug 2026 Official leaderboard
68 Gemma 4 31B IT 50.2 Apr 2026 Official leaderboard
69 Qwen3.5 397B-A17B 49.7 Feb 2026 Official leaderboard
70 Claude Opus 4 49.5 May 2025 Official leaderboard
71 Qwen3.7-Plus 49.4 Jun 2026 Official leaderboard
72 GPT-5.4 Nano 49.0 Mar 2026 Official leaderboard
73 Qwen3-Max 48.6 thinking Sep 2025 Official leaderboard
74 DeepSeek-V4-Flash 48.5 Apr 2026 Official leaderboard
75 GPT-5 mini 48.3 Aug 2025 Official leaderboard
76 Claude Opus 4.1 48.3 Aug 2025 Official leaderboard
77 GPT-5.3 Instant 48.1 Mar 2026 Official leaderboard
78 Gemini 2.5 Pro Jun 2025 47.5 Jun 2025 Official leaderboard
79 Qwen3.5-27B 46.6 Feb 2026 Official leaderboard
80 Qwen 3.5 Plus (hosted 397B-A17B) 46.5 Feb 2026 Official leaderboard
81 MiniMax-M3 46.4 Jun 2026 Official leaderboard
82 Gemini 3.1 Flash-Lite 46.1 Mar 2026 Official leaderboard
83 DeepSeek-V3.2-Exp 45.7 Sep 2025 Official leaderboard
84 Qwen3.5-122B-A10B 45.7 Feb 2026 Official leaderboard
85 Qwen 3.6 Plus 45.7 Mar 2026 Official leaderboard
86 DeepSeek-V3.2 45.4 Dec 2025 Official leaderboard
87 Qwen3.6 27B 45.2 Apr 2026 Official leaderboard
88 MiMo-V2.5-Pro 44.2 Apr 2026 Official leaderboard
89 Qwen 3.6 Flash 43.3 Apr 2026 Official leaderboard
90 Claude Haiku 4.5 43.1 Oct 2025 Official leaderboard
91 Qwen3-235B-A22B-Thinking Jul 2025 42.4 Jul 2025 Official leaderboard
92 Qwen 3.5 Flash (hosted 35B-A3B) 42.3 Feb 2026 Official leaderboard
93 Qwen3.5-35B-A3B 42.1 Feb 2026 Official leaderboard
94 DeepSeek-V3.1-Terminus 42.0 Sep 2025 Official leaderboard
95 Qwen Plus Apr 2025 41.3 Apr 2025 Official leaderboard
96 o4-mini 41.2 Apr 2025 Official leaderboard
97 Claude Sonnet 4 41.1 May 2025 Official leaderboard
98 Gemma 4 26B A4B 40.3 Apr 2026 Official leaderboard
99 DeepSeek-V3.1 39.9 non-thinking Aug 2025 Official leaderboard
100 Qwen 3.6 35B-A3B 39.7 Apr 2026 Official leaderboard
101 gpt-oss-120b 38.9 Aug 2025 Official leaderboard
102 Qwen3 235B-A22B (2507) 38.7 Jul 2025 Official leaderboard
103 o1 37.8 Dec 2024 Official leaderboard
104 Gemini 2.5 Flash Jun 2025 37.7 Jun 2025 Official leaderboard
105 Qwen3-Next-80B-A3B 36.7 thinking Sep 2025 Official leaderboard
106 Mistral Medium 3.5 36.6 Apr 2026 Official leaderboard
107 Qwen3-235B-A22B 36.3 Apr 2025 Official leaderboard
108 GPT-4.1 36.0 Apr 2025 Official leaderboard
109 Qwen3.5-9B 35.1 Feb 2026 Official leaderboard
110 Nemotron 3.5 Lightning 30B-A3B 33.6 Aug 2026 Official leaderboard
111 Qwen3-30B-A3B-Thinking Jul 2025 32.9 Jul 2025 Official leaderboard
112 o3-mini 32.3 Jan 2025 Official leaderboard
113 Qwen3-30B-A3B-Instruct Jul 2025 31.6 Jul 2025 Official leaderboard
114 GPT-4.1 mini 31.0 Apr 2025 Official leaderboard
115 Mistral Small 4 30.5 Mar 2026 Official leaderboard
116 gpt-oss-20b 30.4 Aug 2025 Official leaderboard
117 Qwen3-32B 29.2 Apr 2025 Official leaderboard
118 Qwen3-14B 28.6 Apr 2025 Official leaderboard
119 Mistral Medium 3.1 28.4 Aug 2025 Official leaderboard
120 Mistral Large 3 28.1 Dec 2025 Official leaderboard
121 GPT-4o May 2024 27.1 May 2024 Official leaderboard
122 Llama 4 Maverick 26.8 Apr 2025 Official leaderboard
123 DeepSeek-V3 Mar 2025 26.2 Mar 2025 Official leaderboard
124 Llama 3.3 70B 26.1 Dec 2024 Official leaderboard
125 GPT-4 Jun 2023 25.3 Jun 2023 Official leaderboard
126 Qwen3-30B-A3B 25.2 Apr 2025 Official leaderboard
127 Gemini 2.5 Flash-Lite Jun 2025 25.1 Jun 2025 Official leaderboard
128 DeepSeek Chat 25.0 Dec 2024 Official leaderboard
129 Llama 3.1-70B 24.2 Jul 2024 Official leaderboard
130 Qwen2.5-72B 24.2 Sep 2024 Official leaderboard
131 GPT-5 nano 24.1 Aug 2025 Official leaderboard
132 Claude 3 Opus 24.0 Feb 2024 Official leaderboard
133 Llama 4 Scout 23.1 Apr 2025 Official leaderboard
134 GPT-4 Turbo Apr 2024 21.5 Apr 2024 Official leaderboard
135 Cohere Command A 20.7 Mar 2025 Official leaderboard
136 Qwen3-8B 20.2 Apr 2025 Official leaderboard
137 Gemma 3 27B 19.6 Mar 2025 Official leaderboard
138 Ministral 3 14B 19.3 Dec 2025 Official leaderboard
139 GPT-4o mini 17.5 Jul 2024 Official leaderboard
140 Ministral 3 8B Instruct 17.4 Dec 2025 Official leaderboard
141 Llama 3.1-8B 13.6 Jul 2024 Official leaderboard
142 Gemma 3n E4B 13.3 Jun 2025 Official leaderboard
143 Gemma 3 12B 13.3 Mar 2025 Official leaderboard
144 GPT-3.5 Turbo Nov 2023 12.7 Nov 2023 Official leaderboard
145 GPT-4.1 nano 12.3 Apr 2025 Official leaderboard
146 Qwen2.5-7B 11.9 Sep 2024 Official leaderboard
147 Claude 3 Haiku 11.6 Mar 2024 Official leaderboard
148 Command R+ 11.6 Aug 2024 Official leaderboard
149 c4ai-command-r-08-2024 10.3 Aug 2024 Official leaderboard
150 Ministral 3B 10.1 Oct 2024 Official leaderboard
151 Gemma 3 4B 10.1 Mar 2025 Official leaderboard
152 Command R7B 7.1 Dec 2024 Official leaderboard

3 models have held number one since its release in Aug 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 3 leaders
Claude Opus 5.5 77.9

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0 20 40 60 80 2023 2024 2025 2026 0 20 40 60 80 2023 2024 2025 2026

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Other

All 41 in Other
  • Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
  • Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Gemini 4 Carbon 59.0
  • Anthropic ECI (AECI) Only in system cards Index that combines a basket of capability benchmarks into one capability score to track trends across Anthropic models. Claude Opus 5.5 169.4
  • Qwen in-house English automatic evaluation Only in system cards Models answer Qwen in-house English datasets on knowledge, comprehension, coding, math and related skills. Skills and weights change between reports. Qwen2-72B 72.9%
  • Recognition of model-written user turns (Anthropic) Only in system cards Models write synthetic user turns for real Claude Code transcripts, then judge which version is real. Score is accuracy above chance. Claude Mythos Preview 0.35
  • LiveBench Dropped Models answer monthly refreshed questions across math, coding, reasoning, language, instruction following, and data analysis. Scored automatically against objective ground truth. Claude Fable 5.1 85.7%