Benchmark Almanac

CMIMC 2025

DeepSeek-V3.2-Speciale 95.6% leads since Dec 2025 Not in model cards

Models answer 40 final-answer problems from the 2025 Carnegie Mellon Informatics and Mathematics Competition. MathArena scores the average accuracy over four attempts.

The contest was held in March 2025 and MathArena added it in May 2025. MathArena ran 36 models and later marked the set deprecated.

Made by Carnegie Mellon University (contest) , MathArena (evaluation) Released May 2025 Competition problems with answers All tasks public Last model added Feb 2026 Scores refreshed 2 Nov 2026
Level
Competition The strongest students, in olympiad-style contests
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
eth-sri/matharena 294 stars Paper Dataset Leaderboard cmimc.math.cmu.edu
33
Models scored on its own leaderboard

3 models share the top score of 95.6% among models released by Feb 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in Feb 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

33 models. Scroll sideways for the rest.

  1. 95.6 DeepSeek-V3.2-Speciale
  2. 95.6 GPT-5.2
  3. 95.6 Step 3.5 Flash
  4. 94.4 GPT-5.1
  5. 93.1 GLM-5
  6. 90.6 Kimi K2 Thinking
  7. 90.6 Kimi K2.5
  8. 90.0 Gemini 3 Pro
  9. 90.0 Gemini 3 Flash
  10. 88.8 GPT-5
  11. 87.5 GLM-4.6
  12. 85.0 Grok 4
  13. 85.0 gpt-oss-120b
  14. 85.0 Grok 4 Fast
  15. 84.4 GPT-5 mini
  16. 84.4 Grok 4.1 Fast
  17. 84.4 DeepSeek-V3.2
  18. 83.8 o4-mini
  19. 82.5 o3
  20. 82.5 DeepSeek-V3.1
  21. 76.3 DeepSeek-V3.2-Exp
  22. 74.4 GPT-5 nano
  23. 73.8 gpt-oss-20b
  24. 72.5 GLM-4.5
  25. 71.9 GLM-4.5-Air
  26. 71.9 Falcon H1R 7B
  27. 71.3 DeepSeek-R1
  28. 68.1 Grok-3 mini
  29. 68.1 K2-Think
  30. 66.3 Claude Sonnet 4.5
  31. 61.9 QED-Nano
  32. 58.1 Gemini 2.5 Pro
  33. 54.4 Gemini 2.5 Flash
  1. 1 DeepSeek-V3.2-Speciale 95.6%
  2. 2 GPT-5.2 high 95.6%
  3. 3 Step 3.5 Flash 95.6%
  4. 4 GPT-5.1 high 94.4%
  5. 5 GLM-5 93.1%
  6. 6 Kimi K2 Thinking 90.6%
  7. 7 Kimi K2.5 thinking 90.6%
  8. 8 Gemini 3 Pro 90%
  9. 9 Gemini 3 Flash 90%
  10. 10 GPT-5 high 88.8%
  11. 11 GLM-4.6 87.5%
  12. 12 Grok 4 85%
  13. 13 gpt-oss-120b high 85%
  14. 14 Grok 4 Fast 85%
  15. 15 GPT-5 mini high 84.4%
  16. 16 Grok 4.1 Fast 84.4%
  17. 17 DeepSeek-V3.2 thinking 84.4%
  18. 18 o4-mini high 83.8%
  19. 19 o3 high 82.5%
  20. 20 DeepSeek-V3.1 thinking 82.5%
  21. 21 DeepSeek-V3.2-Exp thinking 76.3%
  22. 22 GPT-5 nano high 74.4%
  23. 23 gpt-oss-20b high 73.8%
  24. 24 GLM-4.5 72.5%
  25. 25 GLM-4.5-Air 71.9%
  26. 26 Falcon H1R 7B 71.9%
  27. 27 DeepSeek-R1 May 2025 71.3%
  28. 28 Grok-3 mini high 68.1%
  29. 29 K2-Think 68.1%
  30. 30 Claude Sonnet 4.5 thinking 66.3%
  31. 31 QED-Nano 61.9%
  32. 32 Gemini 2.5 Pro Mar 2025 58.1%
  33. 33 Gemini 2.5 Flash Sep 2025, thinking 54.4%
#ModelScoreSettingDateSource
1 DeepSeek-V3.2-Speciale 95.6% Dec 2025 Official leaderboard
2 GPT-5.2 95.6% high Dec 2025 Official leaderboard
3 Step 3.5 Flash 95.6% Feb 2026 Official leaderboard
4 GPT-5.1 94.4% high Nov 2025 Official leaderboard
5 GLM-5 93.1% Feb 2026 Official leaderboard
6 Kimi K2 Thinking 90.6% Nov 2025 Official leaderboard
7 Kimi K2.5 90.6% thinking Jan 2026 Official leaderboard
8 Gemini 3 Pro 90% Nov 2025 Official leaderboard
9 Gemini 3 Flash 90% Dec 2025 Official leaderboard
10 GPT-5 88.8% high Aug 2025 Official leaderboard
11 GLM-4.6 87.5% Sep 2025 Official leaderboard
12 Grok 4 85% Jul 2025 Official leaderboard
13 gpt-oss-120b 85% high Aug 2025 Official leaderboard
14 Grok 4 Fast 85% Sep 2025 Official leaderboard
15 GPT-5 mini 84.4% high Aug 2025 Official leaderboard
16 Grok 4.1 Fast 84.4% Nov 2025 Official leaderboard
17 DeepSeek-V3.2 84.4% thinking Dec 2025 Official leaderboard
18 o4-mini 83.8% high Apr 2025 Official leaderboard
19 o3 82.5% high Apr 2025 Official leaderboard
20 DeepSeek-V3.1 82.5% thinking Aug 2025 Official leaderboard
21 DeepSeek-V3.2-Exp 76.3% thinking Sep 2025 Official leaderboard
22 GPT-5 nano 74.4% high Aug 2025 Official leaderboard
23 gpt-oss-20b 73.8% high Aug 2025 Official leaderboard
24 GLM-4.5 72.5% Aug 2025 Official leaderboard
25 GLM-4.5-Air 71.9% Jul 2025 Official leaderboard
26 Falcon H1R 7B 71.9% Jan 2026 Official leaderboard
27 DeepSeek-R1 May 2025 71.3% May 2025 Official leaderboard
28 Grok-3 mini 68.1% high Apr 2025 Official leaderboard
29 K2-Think 68.1% Sep 2025 Official leaderboard
30 Claude Sonnet 4.5 66.3% thinking Sep 2025 Official leaderboard
31 QED-Nano 61.9% Feb 2026 Official leaderboard
32 Gemini 2.5 Pro Mar 2025 58.1% Mar 2025 Official leaderboard
33 Gemini 2.5 Flash Sep 2025 54.4% thinking Sep 2025 Official leaderboard

The best score, 95.6%, costs $0.01 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

xAI DeepSeek Technology Innovation Institute Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 Cost per task, log scale DeepSeek-V3.2-Speciale Grok 4 Fast Falcon H1R 7B Grok-3 mini, low 0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 Cost per task, log scale DeepSeek-V3.2-Speciale Grok 4 Fast
ModelEffortHarnessScoreCost per taskOutput tokens
DeepSeek-V3.2-Speciale 95.6% $0.01 29,184
GPT-5.2 high 95.6% $0.14 9,923
Step 3.5 Flash 95.6% $0.01 47,208
GPT-5.1 high 94.4% $0.23 23,435
GLM-5 93.1% $0.11 34,178
Kimi K2 Thinking 90.6% $0.07 26,190
Kimi K2.5 thinking 90.6% $0.09 30,844
Gemini 3 Pro 90% $0.20 17,005
Gemini 3 Flash 90% $0.07 22,345
GPT-5 high 88.8% $0.17 17,108
GLM-4.6 87.5% $0.05 21,111
Grok 4 85% $0.31 20,365
gpt-oss-120b high 85% $0.02 26,116
Grok 4 Fast 85% $0.007 14,479
GPT-5 mini high 84.4% $0.04 19,425
Grok 4.1 Fast 84.4% $0.007 14,764
DeepSeek-V3.2 thinking 84.4% $0.008 19,963
o4-mini high 83.8% $0.05 9,066
o3 high 82.5% $0.10 11,054
DeepSeek-V3.1 thinking 82.5% $0.05 21,023
DeepSeek-V3.2-Exp thinking 76.3% $0.009 20,543
GPT-5 nano high 74.4% $0.01 32,245
gpt-oss-20b high 73.8% $0.01 49,384
GLM-4.5 72.5% $0.06 27,139
GLM-4.5-Air 71.9% $0.03 27,722
Falcon H1R 7B 71.9% $0.006 29,902
DeepSeek-R1 May 2025 71.3% $0.06 25,526
Grok-3 mini high 68.1% $0.01 27,686
K2-Think 68.1%
Claude Sonnet 4.5 thinking 66.3% $0.30 20,159
o4-mini medium 62.5% $0.03 6,983
QED-Nano 61.9% 51,289
Gemini 2.5 Pro Mar 2025 58.1% $0.17 17,005
Gemini 2.5 Flash Sep 2025 thinking 54.4% $0.07 21,464
o4-mini low 48.1% $0.01 2,604
Grok-3 mini low 40% $0.004 7,056

6 models have held number one since its release in May 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI xAI Moonshot AI DeepSeek
Own leaderboard Passed 85% in Jul 2025, by Grok 4
DeepSeek-V3.2-Speciale 95.6%

The best score passed 85% in Jul 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) DeepSeek-V3.2-Speciale Kimi K2 Thinking GPT-5 o4-mini Grok-3 mini Gemini 2.5 Pro 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) DeepSeek-V3.2-Speciale GPT-5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%