Benchmark Almanac

Extended NYT Connections

Gemini 4 Argon 98.6% leads since Sep 2026 Not in model cards

Models solve New York Times Connections puzzles with up to four extra trick words added. Each puzzle scores by the square of the share of groups found.

The puzzle set grows over time, now 940 puzzles. Trend charts use a shared 601-puzzle set so older runs stay comparable.

Made by Lech Mazur Released Oct 2024 Language reasoning All tasks public Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
School Students, from grade school to high school
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
lechmazur/nyt-connections 250 stars Leaderboard
101
Entries on its own leaderboard, 97 models we track

Gemini 4 Argon leads with 98.6%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

101 entries. Scroll sideways for the rest.

  1. 98.6 Gemini 4 Argon
  2. 98.4 GPT-6 Astra
  3. 97.8 GPT-6.1 Sol
  4. 97.4 Gemini 3.1 Pro
  5. 97.4 Gemini 3.8 Flash
  6. 96.2 GPT-5.5
  7. 94.4 Gemini 3 Pro
  8. 94.3 Claude Opus 5
  9. 94.0 Gemini 3.7 Flash
  10. 93.8 GPT-5.6 Sol
  11. 93.6 Kimi K3
  12. 93.4 Muse Wonder
  13. 93.4 Kimi K3.1 Pro
  14. 92.8 DeepSeek V4.1 Flash 1017
  15. 92.7 Claude Fable 5
  16. 92.6 Gemini 3.5 Flash
  17. 92.1 Claude Opus 4.6
  18. 91.3 GPT-5.4
  19. 91.3 DeepSeek-V4-Pro
  20. 91.1 Claude Opus 4.8
  21. 90.1 GPT-6 Sol
  22. 90.0 Claude Fable 5.1
  23. 89.6 Grok 4.20 Multi-Agent Exp Beta 0304
  24. 89.6 DeepSeek-V4-Flash
  25. 89.6 DeepSeek V4.1 Flash
  26. 89.0 Gemini 3.6 Flash
  27. 88.5 Claude Opus 5.5
  28. 88.3 Qwen 3.8 Max
  29. 87.4 Grok 4.1 Fast
  30. 87.2 Kimi K2.6
  31. 85.4 Grok 4.20
  32. 85.1 Qwen3.7-Max
  33. 85.1 Muse Spark 1.3
  34. 84.9 Muse Spark 1.1
  35. 83.7 Grok 4.20 Reasoning Exp Beta 0304
  36. 83.6 GPT-5.2
  37. 83.1 Gemini 3 Flash
  38. 80.9 Claude Sonnet 4.6
  39. 80.5 Claude Sonnet 5.5
  40. 80.0 Grok 4.6
  41. 79.9 Grok 4.5
  42. 79.3 GPT-5.2 Pro
  43. 79.2 Muse Spark 1.2
  44. 78.4 GPT-5.6 Terra
  45. 77.7 GLM-5.1
  46. 76.8 Grok 4.7
  47. 75.1 Claude Sonnet 5
  48. 74.8 GLM-5
  49. 74.8 Qwen3.7-Plus
  50. 74.3 GLM-5.2
  51. 74.2 GLM-5.3
  52. 74.1 Qwen 3.6 Max (Preview)
  53. 70.6 Gemma 4 31B IT
  54. 69.9 Kimi K2.5
  55. 69.4 GPT-5.6 Luna
  56. 68.7 GPT-6 Luna
  57. 68.2 Tencent Hy4 preview
  58. 65.7 Claude Haiku 5.5
  59. 65.1 MiniMax-M3
  60. 61.8 GPT-5.4 Mini
  61. 60.4 Gemini 3.5 Flash-Lite
  62. 60.3 Qwen 3.6 Plus
  63. 58.9 Qwen3.5 397B-A17B
  64. 55.2 Grok 4.3
  65. 54.5 Qwen 3.8 27B
  66. 52.5 Claude Opus 4.5
  67. 51.7 Qwen3.5-122B-A10B
  68. 47.9 Qwen3.5-27B
  69. 43.8 Qwen3.7 Flash
  70. 41.6 Qwen 3.6 35B-A3B
  71. 41.2 Tencent Hy3
  72. 39.7 Step 3.7 Flash
  73. 39.0 Claude Opus 4.7
  74. 37.3 Claude Sonnet 4.5
  75. 36.7 DeepSeek-V3.2
  76. 34.4 MiMo-V2.5-Pro
  77. 30.1 Qwen3 Max (2026-01-23)
  78. 28.4 Step 3.5 Flash
  79. 28.4 Seed2.0 Pro
  80. 28.3 MiniMax-M3.1-Flash
  81. 27.4 Mistral Large 4
  82. 25.8 MiMo-V2-Pro
  83. 24.7 MiniMax-M2.7
  84. 23.4 ERNIE 5.1
  85. 17.7 LongCat-Flash-Thinking
  86. 17.2 Tencent Hy3 preview
  87. 16.8 MiniMax-M2.5
  88. 16.5 Trinity Large Thinking
  89. 15.4 Nemotron 3 Super
  90. 14.8 MiniMax-M2
  91. 14.3 Claude Haiku 4.5
  92. 12.9 Mistral Medium 3.5
  93. 11.8 Qwen3-Max
  94. 11.2 MiniMax-M2.1
  95. 10.3 ERNIE 5.0
  96. 8.20 Gemini 3.1 Flash-Lite
  97. 8.00 Llama 4 Maverick
  98. 7.60 Grok 4.20 Non-Reasoning Exp Beta 0304
  99. 7.50 Mistral Large 3
  100. 6.50 Mistral Medium 3.1
  101. 4.10 Ling-2.6-1T
  1. 1 Gemini 4 Argon high 98.6%
  2. 2 GPT-6 Astra max 98.4%
  3. 3 GPT-6.1 Sol max 97.8%
  4. 4 Gemini 3.1 Pro 97.4%
  5. 5 Gemini 3.8 Flash high 97.4%
  6. 6 GPT-5.5 xhigh 96.2%
  7. 7 Gemini 3 Pro 94.4%
  8. 8 Claude Opus 5 xhigh 94.3%
  9. 9 Gemini 3.7 Flash 94%
  10. 10 GPT-5.6 Sol xhigh 93.8%
  11. 11 Kimi K3 93.6%
  12. 12 Muse Wonder high 93.4%
  13. 13 Kimi K3.1 Pro high 93.4%
  14. 14 DeepSeek V4.1 Flash 1017 high 92.8%
  15. 15 Claude Fable 5 high 92.7%
  16. 16 Gemini 3.5 Flash 92.6%
  17. 17 Claude Opus 4.6 high 92.1%
  18. 18 GPT-5.4 xhigh 91.3%
  19. 19 DeepSeek-V4-Pro high 91.3%
  20. 20 Claude Opus 4.8 xhigh 91.1%
  21. 21 GPT-6 Sol high 90.1%
  22. 22 Claude Fable 5.1 high 90%
  23. 23 Grok 4.20 Multi-Agent Exp Beta 0304 89.6%
  24. 24 DeepSeek-V4-Flash 89.6%
  25. 25 DeepSeek V4.1 Flash high 89.6%
  26. 26 Gemini 3.6 Flash 89%
  27. 27 Claude Opus 5.5 high 88.5%
  28. 28 Qwen 3.8 Max 88.3%
  29. 29 Grok 4.1 Fast reasoning 87.4%
  30. 30 Kimi K2.6 87.2%
  31. 31 Grok 4.20 reasoning 85.4%
  32. 32 Qwen3.7-Max 85.1%
  33. 33 Muse Spark 1.3 high 85.1%
  34. 34 Muse Spark 1.1 high 84.9%
  35. 35 Grok 4.20 Reasoning Exp Beta 0304 83.7%
  36. 36 GPT-5.2 xhigh 83.6%
  37. 37 Gemini 3 Flash 83.1%
  38. 38 Claude Sonnet 4.6 high 80.9%
  39. 39 Claude Sonnet 5.5 high 80.5%
  40. 40 Grok 4.6 xhigh 80%
  41. 41 Grok 4.5 high 79.9%
  42. 42 GPT-5.2 Pro 79.3%
  43. 43 Muse Spark 1.2 high 79.2%
  44. 44 GPT-5.6 Terra high 78.4%
  45. 45 GLM-5.1 77.7%
  46. 46 Grok 4.7 high 76.8%
  47. 47 Claude Sonnet 5 high 75.1%
  48. 48 GLM-5 74.8%
  49. 49 Qwen3.7-Plus 74.8%
  50. 50 GLM-5.2 high 74.3%
  51. 51 GLM-5.3 high 74.2%
  52. 52 Qwen 3.6 Max (Preview) 74.1%
  53. 53 Gemma 4 31B IT reasoning 70.6%
  54. 54 Kimi K2.5 thinking 69.9%
  55. 55 GPT-5.6 Luna high 69.4%
  56. 56 GPT-6 Luna high 68.7%
  57. 57 Tencent Hy4 preview 68.2%
  58. 58 Claude Haiku 5.5 high 65.7%
  59. 59 MiniMax-M3 65.1%
  60. 60 GPT-5.4 Mini xhigh 61.8%
  61. 61 Gemini 3.5 Flash-Lite high 60.4%
  62. 62 Qwen 3.6 Plus 60.3%
  63. 63 Qwen3.5 397B-A17B 58.9%
  64. 64 Grok 4.3 55.2%
  65. 65 Qwen 3.8 27B 54.5%
  66. 66 Claude Opus 4.5 thinking 16K 52.5%
  67. 67 Qwen3.5-122B-A10B 51.7%
  68. 68 Qwen3.5-27B 47.9%
  69. 69 Qwen3.7 Flash 43.8%
  70. 70 Qwen 3.6 35B-A3B 41.6%
  71. 71 Tencent Hy3 high 41.2%
  72. 72 Step 3.7 Flash high 39.7%
  73. 73 Claude Opus 4.7 high 39%
  74. 74 Claude Sonnet 4.5 thinking 16K 37.3%
  75. 75 DeepSeek-V3.2 36.7%
  76. 76 MiMo-V2.5-Pro 34.4%
  77. 77 Qwen3 Max (2026-01-23) 30.1%
  78. 78 Step 3.5 Flash 28.4%
  79. 79 Seed2.0 Pro 28.4%
  80. 80 MiniMax-M3.1-Flash high 28.3%
  81. 81 Mistral Large 4 high 27.4%
  82. 82 MiMo-V2-Pro 25.8%
  83. 83 MiniMax-M2.7 24.7%
  84. 84 ERNIE 5.1 23.4%
  85. 85 LongCat-Flash-Thinking 17.7%
  86. 86 Tencent Hy3 preview 17.2%
  87. 87 MiniMax-M2.5 16.8%
  88. 88 Trinity Large Thinking 16.5%
  89. 89 Nemotron 3 Super 15.4%
  90. 90 MiniMax-M2 14.8%
  91. 91 Claude Haiku 4.5 14.3%
  92. 92 Mistral Medium 3.5 high 12.9%
  93. 93 Qwen3-Max 11.8%
  94. 94 MiniMax-M2.1 11.2%
  95. 95 ERNIE 5.0 10.3%
  96. 96 Gemini 3.1 Flash-Lite 8.20%
  97. 97 Llama 4 Maverick 8.00%
  98. 98 Grok 4.20 Non-Reasoning Exp Beta 0304 7.60%
  99. 99 Mistral Large 3 7.50%
  100. 100 Mistral Medium 3.1 6.50%
  101. 101 Ling-2.6-1T 4.10%
#EntryScoreSettingDateSource
1 Gemini 4 Argon 98.6% high Sep 2026 Official leaderboard
2 GPT-6 Astra 98.4% max Sep 2026 Official leaderboard
3 GPT-6.1 Sol 97.8% max Sep 2026 Official leaderboard
4 Gemini 3.1 Pro 97.4% Feb 2026 Official leaderboard
5 Gemini 3.8 Flash 97.4% high Sep 2026 Official leaderboard
6 GPT-5.5 96.2% xhigh Apr 2026 Official leaderboard
7 Gemini 3 Pro 94.4% Nov 2025 Official leaderboard
8 Claude Opus 5 94.3% xhigh Jul 2026 Official leaderboard
9 Gemini 3.7 Flash 94% Aug 2026 Official leaderboard
10 GPT-5.6 Sol 93.8% xhigh Jul 2026 Official leaderboard
11 Kimi K3 93.6% Jul 2026 Official leaderboard
12 Muse Wonder 93.4% high Oct 2026 Official leaderboard
13 Kimi K3.1 Pro 93.4% high Oct 2026 Official leaderboard
14 DeepSeek V4.1 Flash 1017 92.8% high Oct 2026 Official leaderboard
15 Claude Fable 5 92.7% high Jun 2026 Official leaderboard
16 Gemini 3.5 Flash 92.6% May 2026 Official leaderboard
17 Claude Opus 4.6 92.1% high Feb 2026 Official leaderboard
18 GPT-5.4 91.3% xhigh Mar 2026 Official leaderboard
19 DeepSeek-V4-Pro 91.3% high Apr 2026 Official leaderboard
20 Claude Opus 4.8 91.1% xhigh May 2026 Official leaderboard
21 GPT-6 Sol 90.1% high Sep 2026 Official leaderboard
22 Claude Fable 5.1 90% high Sep 2026 Official leaderboard
23 Grok 4.20 Multi-Agent Exp Beta 0304 89.6% Official leaderboard
24 DeepSeek-V4-Flash 89.6% Apr 2026 Official leaderboard
25 DeepSeek V4.1 Flash 89.6% high Sep 2026 Official leaderboard
26 Gemini 3.6 Flash 89% Jul 2026 Official leaderboard
27 Claude Opus 5.5 88.5% high Sep 2026 Official leaderboard
28 Qwen 3.8 Max 88.3% Aug 2026 Official leaderboard
29 Grok 4.1 Fast 87.4% reasoning Nov 2025 Official leaderboard
30 Kimi K2.6 87.2% Apr 2026 Official leaderboard
31 Grok 4.20 85.4% reasoning Feb 2026 Official leaderboard
32 Qwen3.7-Max 85.1% May 2026 Official leaderboard
33 Muse Spark 1.3 85.1% high Sep 2026 Official leaderboard
34 Muse Spark 1.1 84.9% high Jul 2026 Official leaderboard
35 Grok 4.20 Reasoning Exp Beta 0304 83.7% Official leaderboard
36 GPT-5.2 83.6% xhigh Dec 2025 Official leaderboard
37 Gemini 3 Flash 83.1% Dec 2025 Official leaderboard
38 Claude Sonnet 4.6 80.9% high Feb 2026 Official leaderboard
39 Claude Sonnet 5.5 80.5% high Sep 2026 Official leaderboard
40 Grok 4.6 80% xhigh Aug 2026 Official leaderboard
41 Grok 4.5 79.9% high Jul 2026 Official leaderboard
42 GPT-5.2 Pro 79.3% Dec 2025 Official leaderboard
43 Muse Spark 1.2 79.2% high Aug 2026 Official leaderboard
44 GPT-5.6 Terra 78.4% high Jul 2026 Official leaderboard
45 GLM-5.1 77.7% Apr 2026 Official leaderboard
46 Grok 4.7 76.8% high Sep 2026 Official leaderboard
47 Claude Sonnet 5 75.1% high Jun 2026 Official leaderboard
48 GLM-5 74.8% Feb 2026 Official leaderboard
49 Qwen3.7-Plus 74.8% Jun 2026 Official leaderboard
50 GLM-5.2 74.3% high Jun 2026 Official leaderboard
51 GLM-5.3 74.2% high Aug 2026 Official leaderboard
52 Qwen 3.6 Max (Preview) 74.1% Apr 2026 Official leaderboard
53 Gemma 4 31B IT 70.6% reasoning Apr 2026 Official leaderboard
54 Kimi K2.5 69.9% thinking Jan 2026 Official leaderboard
55 GPT-5.6 Luna 69.4% high Jul 2026 Official leaderboard
56 GPT-6 Luna 68.7% high Sep 2026 Official leaderboard
57 Tencent Hy4 preview 68.2% Aug 2026 Official leaderboard
58 Claude Haiku 5.5 65.7% high Oct 2026 Official leaderboard
59 MiniMax-M3 65.1% Jun 2026 Official leaderboard
60 GPT-5.4 Mini 61.8% xhigh Mar 2026 Official leaderboard
61 Gemini 3.5 Flash-Lite 60.4% high Jul 2026 Official leaderboard
62 Qwen 3.6 Plus 60.3% Mar 2026 Official leaderboard
63 Qwen3.5 397B-A17B 58.9% Feb 2026 Official leaderboard
64 Grok 4.3 55.2% Apr 2026 Official leaderboard
65 Qwen 3.8 27B 54.5% Aug 2026 Official leaderboard
66 Claude Opus 4.5 52.5% thinking 16K Nov 2025 Official leaderboard
67 Qwen3.5-122B-A10B 51.7% Feb 2026 Official leaderboard
68 Qwen3.5-27B 47.9% Feb 2026 Official leaderboard
69 Qwen3.7 Flash 43.8% Jul 2026 Official leaderboard
70 Qwen 3.6 35B-A3B 41.6% Apr 2026 Official leaderboard
71 Tencent Hy3 41.2% high Jul 2026 Official leaderboard
72 Step 3.7 Flash 39.7% high May 2026 Official leaderboard
73 Claude Opus 4.7 39% high Apr 2026 Official leaderboard
74 Claude Sonnet 4.5 37.3% thinking 16K Sep 2025 Official leaderboard
75 DeepSeek-V3.2 36.7% Dec 2025 Official leaderboard
76 MiMo-V2.5-Pro 34.4% Apr 2026 Official leaderboard
77 Qwen3 Max (2026-01-23) 30.1% Official leaderboard
78 Step 3.5 Flash 28.4% Feb 2026 Official leaderboard
79 Seed2.0 Pro 28.4% Feb 2026 Official leaderboard
80 MiniMax-M3.1-Flash 28.3% high Oct 2026 Official leaderboard
81 Mistral Large 4 27.4% high Oct 2026 Official leaderboard
82 MiMo-V2-Pro 25.8% Mar 2026 Official leaderboard
83 MiniMax-M2.7 24.7% Mar 2026 Official leaderboard
84 ERNIE 5.1 23.4% May 2026 Official leaderboard
85 LongCat-Flash-Thinking 17.7% Sep 2025 Official leaderboard
86 Tencent Hy3 preview 17.2% Apr 2026 Official leaderboard
87 MiniMax-M2.5 16.8% Feb 2026 Official leaderboard
88 Trinity Large Thinking 16.5% Apr 2026 Official leaderboard
89 Nemotron 3 Super 15.4% Mar 2026 Official leaderboard
90 MiniMax-M2 14.8% Oct 2025 Official leaderboard
91 Claude Haiku 4.5 14.3% Oct 2025 Official leaderboard
92 Mistral Medium 3.5 12.9% high Apr 2026 Official leaderboard
93 Qwen3-Max 11.8% Sep 2025 Official leaderboard
94 MiniMax-M2.1 11.2% Dec 2025 Official leaderboard
95 ERNIE 5.0 10.3% Jan 2026 Official leaderboard
96 Gemini 3.1 Flash-Lite 8.20% Mar 2026 Official leaderboard
97 Llama 4 Maverick 8.00% Apr 2025 Official leaderboard
98 Grok 4.20 Non-Reasoning Exp Beta 0304 7.60% Official leaderboard
99 Mistral Large 3 7.50% Dec 2025 Official leaderboard
100 Mistral Medium 3.1 6.50% Aug 2025 Official leaderboard
101 Ling-2.6-1T 4.10% Apr 2026 Official leaderboard

More effort adds a median 2 points

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
GPT-6 Astra
high
98.4%
GPT-6.1 Sol
high
97.8%
GPT-5.5
no reasoning medium
96.2%
Claude Opus 5
high
94.3%
GPT-5.6 Sol
high
93.8%
Claude Opus 4.6
no reasoning thinking 16K high
92.1%
GPT-5.4
no reasoning medium
91.3%
DeepSeek-V4-Pro
high
91.3%
Claude Opus 4.8
high
91.1%
Claude Opus 5.5
medium
88.5%
Grok 4.1 Fast
non-reasoni… reasoning
87.4%
Grok 4.20
non-reasoni… reasoning
85.4%
ModelEffortHarnessScoreCost per taskOutput tokens
Gemini 4 Argon high 98.6%
GPT-6 Astra max 98.4%
GPT-6 Astra xhigh 98.1%
GPT-6.1 Sol max 97.8%
GPT-6 Astra high 97.7%
Gemini 3.1 Pro 97.4%
Gemini 3.8 Flash high 97.4%
GPT-5.5 xhigh 96.2%
GPT-6.1 Sol high 95.5%
GPT-5.5 high 95.2%
Gemini 3 Pro 94.4%
Claude Opus 5 xhigh 94.3%
Gemini 3.7 Flash 94%
GPT-5.6 Sol xhigh 93.8%
Kimi K3 93.6%
Muse Wonder high 93.4%
Kimi K3.1 Pro high 93.4%
DeepSeek V4.1 Flash 1017 high 92.8%
Claude Fable 5 high 92.7%
Gemini 3.5 Flash 92.6%
GPT-5.5 medium 92.4%
Claude Opus 5 high 92.2%
Claude Opus 4.6 high 92.1%
GPT-5.4 xhigh 91.3%
DeepSeek-V4-Pro high 91.3%
Claude Opus 4.8 xhigh 91.1%
GPT-5.6 Sol high 91%
GPT-5.4 high 90.6%
GPT-6 Sol high 90.1%
Claude Fable 5.1 high 90%
DeepSeek-V4-Flash 89.6%
DeepSeek V4.1 Flash high 89.6%
Gemini 3.6 Flash 89%
Claude Opus 5.5 high 88.5%
Claude Opus 4.8 high 88.3%
Qwen 3.8 Max 88.3%
GPT-5.4 medium 87.8%
Claude Opus 5.5 medium 87.8%
Grok 4.1 Fast reasoning 87.4%
Kimi K2.6 87.2%
Grok 4.20 reasoning 85.4%
Qwen3.7-Max 85.1%
Muse Spark 1.3 high 85.1%
Muse Spark 1.1 high 84.9%
GPT-5.2 xhigh 83.6%
Gemini 3 Flash 83.1%
Claude Sonnet 4.6 high 80.9%
Claude Sonnet 5.5 high 80.5%
Grok 4.6 xhigh 80%
Grok 4.5 high 79.9%
GPT-5.2 Pro 79.3%
Muse Spark 1.2 high 79.2%
Grok 4.6 high 79%
GPT-5.6 Terra high 78.4%
GLM-5.1 77.7%
Grok 4.7 high 76.8%
Claude Opus 4.6 thinking 16K 76.4%
Claude Sonnet 4.6 thinking 32K 76.4%
Claude Sonnet 5 high 75.1%
GLM-5 74.8%
Qwen3.7-Plus 74.8%
GLM-5.2 high 74.3%
GLM-5.3 high 74.2%
Qwen 3.6 Max (Preview) 74.1%
Gemma 4 31B IT reasoning 70.6%
Kimi K2.5 thinking 69.9%
GPT-5.6 Luna high 69.4%
GPT-6 Luna high 68.7%
Tencent Hy4 preview 68.2%
GPT-5.2 high 68.1%
DeepSeek-V4-Pro 67.3%
Claude Haiku 5.5 high 65.7%
MiniMax-M3 65.1%
GPT-5.4 Mini xhigh 61.8%
GPT-5.2 medium 60.6%
Gemini 3.5 Flash-Lite high 60.4%
Qwen 3.6 Plus 60.3%
Qwen3.5 397B-A17B 58.9%
Grok 4.3 55.2%
Qwen 3.8 27B 54.5%
GPT-5.2 low 54.2%
Claude Opus 4.5 thinking 16K 52.5%
Qwen3.5-122B-A10B 51.7%
Claude Opus 4.5 no reasoning 49.4%
Claude Sonnet 4.6 thinking 16K 48%
Qwen3.5-27B 47.9%
Claude Sonnet 4.6 no reasoning 44.8%
Qwen3.7 Flash 43.8%
Claude Opus 4.6 no reasoning 43.2%
Qwen 3.6 35B-A3B 41.6%
Tencent Hy3 high 41.2%
Step 3.7 Flash high 39.7%
Claude Opus 4.7 high 39%
Claude Sonnet 4.5 thinking 16K 37.3%
DeepSeek-V3.2 36.7%
Claude Sonnet 4.5 no reasoning 35.8%
MiMo-V2.5-Pro 34.4%
Step 3.5 Flash 28.4%
Seed2.0 Pro 28.4%
MiniMax-M3.1-Flash high 28.3%
Mistral Large 4 high 27.4%
MiMo-V2-Pro 25.8%
MiniMax-M2.7 24.7%
ERNIE 5.1 23.4%
GPT-5.5 no reasoning 22%
GPT-5.4 no reasoning 17.8%
LongCat-Flash-Thinking 17.7%
Tencent Hy3 preview 17.2%
MiniMax-M2.5 16.8%
Trinity Large Thinking 16.5%
Gemma 4 31B IT 15.7%
Nemotron 3 Super 15.4%
MiniMax-M2 14.8%
GPT-5.2 no reasoning 14.5%
Claude Haiku 4.5 14.3%
Mistral Medium 3.5 high 12.9%
Grok 4.1 Fast non-reasoning 12.3%
Qwen3-Max 11.8%
MiniMax-M2.1 11.2%
Claude Opus 4.7 no reasoning 10.8%
ERNIE 5.0 10.3%
Grok 4.20 non-reasoning 8.60%
DeepSeek-V3.2 no reasoning 8.20%
Gemini 3.1 Flash-Lite 8.20%
Llama 4 Maverick 8.00%
Mistral Large 3 7.50%
Tencent Hy3 non-thinking 6.90%
Mistral Medium 3.1 6.50%
Ling-2.6-1T 4.10%

7 models have held number one since its release in Oct 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Meta Meituan Anthropic Google DeepMind OpenAI
Own leaderboard Passed 85% in Nov 2025, by Gemini 3 Pro
Gemini 4 Argon 98.6%

The best score passed 85% in Nov 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Gemini 4 Argon Gemini 3.1 Pro Claude Sonnet 4.5 LongCat-Flash-Thinking Llama 4 Maverick 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Gemini 4 Argon Claude Sonnet 4.5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Knowledge

All 109 in Knowledge
  • MMLU-Pro In use Harder 10-option version of MMLU with reasoning-focused questions across 14 subjects. Intern-S2-397B 89.8%
  • SuperGPQA In use 26,529 graduate-level multiple-choice questions across 285 disciplines. Accuracy. Qwen 3.6 Max (Preview) 73.9%
  • WideSearch In use 200 bilingual broad info-seeking tasks: agent collects many atomic facts into a table. Scored by success rate and row/item F1. Tencent Hy4 preview 83.9%
  • KMMLU-Pro In use Models answer multiple-choice questions taken from Korean National Professional Licensure exams. It tests professional knowledge used in Korean industry. Solar Pro 4 79.2%
  • BIG-Bench Extra Hard In use Models solve harder replacements for each of the 23 BIG-Bench Hard tasks, 4,520 examples in total. The score is a harmonic mean of task accuracies. Gemma 4 31B IT 74.4%
  • HLE-Verified In use Verified and revised version of Humanity's Last Exam (1,811 items) with uncertain items removed. Seed2.0 Pro 73.6%