Benchmark Almanac

MathArena (Apex, IMO, USAMO)

Claude Opus 4.8 81.3% leads since May 2026 In use

Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores.

Apex holds 12 problems and Apex Shortlist holds 47, both picked in August 2025. In May 2026 MathArena found no new problems hard enough for an Apex v2. Proof scores for IMO and USAMO depend on judges.

In the authors’ words

The rapid advancement of reasoning capabilities in large language models (LLMs) has led to notable improvements on mathematical benchmarks. However, many of the most commonly used evaluation datasets (e.g., AIME 2024) are widely available online, making it difficult to disentangle genuine reasoning from potential memorization. Furthermore, these benchmarks do not evaluate proof-writing capabilities, which are crucial for many mathematical tasks. To address this, we introduce MathArena, a new benchmark based on the following key insight: recurring math competitions provide a stream of high-quality, challenging problems that can be used for real-time evaluation of LLMs. By evaluating models as soon as new problems are released, we effectively eliminate the risk of contamination. Using this framework, we find strong signs of contamination in AIME 2024. Nonetheless, evaluations on harder competitions, such as CMIMC 2025, demonstrate impressive reasoning capabilities in top-performing models. MathArena is also the first benchmark for proof-writing capabilities. On IMO 2025, top models achieve slightly less than 40%, demonstrating both notable progress and significant room for improvement. So far, we have evaluated over 50 models across seven competitions, totaling 162 problems. As an evolving benchmark, MathArena will continue to track the progress of LLMs on newly released competitions, ensuring rigorous and up-to-date evaluation of mathematical reasoning.

Abstract of MathArena: Evaluating LLMs on Uncontaminated Math Competitions (arXiv:2505.23281, CC0)
Made by ETH Zurich SRI Lab Released Feb 2025 Competition problems with answers All tasks public Last in a launch post Oct 2026 Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Competition The strongest students, in olympiad-style contests
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
47
Models scored on its own leaderboard
12
Launch posts list it, from 6 labs
7
System cards report it, from 3 labs

Claude Opus 4.8 leads with 81.3% among models released by Jul 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in Jul 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Claude Opus 4.8 max 81.3%
  2. 2 GPT-5.5 xhigh 80.2%
  3. 3 GPT-5.4 Pro xhigh 69.8%
  4. 4 Kimi K3 thinking 65.6%
  5. 5 Gemini 3.1 Pro 60.9%
  6. 6 GPT-5.4 xhigh 54.2%
  7. 7 Claude Opus 4.7 xhigh 40.6%
  8. 8 Claude Opus 4.6 high 34.5%
  9. 9 Gemini 3.5 Flash 32.3%
  10. 10 DeepSeek-V4-Pro max 28.1%
  11. 11 DeepSeek-V4-Flash max 27.1%
  12. 12 Gemini 3.6 Flash 26%
  13. 13 Kimi K2.6 thinking 24%
  14. 14 Gemini 3 Pro 23.4%
  15. 15 GLM-5.2 19.8%
  16. 16 Gemini 3 Flash 15.6%
  17. 17 Step 3.7 Flash 14.6%
  18. 18 GPT-5.2 high 13.5%
  19. 19 Step 3.5 Flash 13.5%
  20. 20 GLM-5.1 11.5%
  21. 21 GLM-5 10.9%
  22. 22 DeepSeek-V3.2-Speciale 9.38%
  23. 23 Kimi K2.5 thinking 8.85%
  24. 24 Nemotron 3 Super 7.81%
  25. 25 Qwen3-235B-A22B-Thinking Jul 2025 5.21%
  26. 26 Grok 4 Fast 5.21%
  27. 27 Grok 4.1 Fast 5.21%
  28. 28 Qwen3.5-35B-A3B 4.17%
  29. 29 Grok 4 2.08%
  30. 30 Qwen3-4B-Thinking-2507 2.08%
  31. 31 GPT-5 high 2.08%
  32. 32 DeepSeek-V3.2 thinking 2.08%
  33. 33 Qwen3.5-27B 2.08%
  34. 34 Claude Sonnet 4.5 thinking 1.56%
  35. 35 QED-Nano 1.56%
  36. 36 DeepSeek-R1 May 2025 1.04%
  37. 37 GLM-4.5 1.04%
  38. 38 gpt-oss-120b high 1.04%
  39. 39 GPT-5 mini high 1.04%
  40. 40 GPT-5.1 high 1.04%
  41. 41 Gemini 2.5 Pro Jun 2025 0.52%
  42. 42 Qwen3-30B-A3B-Thinking Jul 2025 0.52%
  43. 43 DeepSeek-V3.1 thinking 0.52%
  44. 44 DeepSeek-V3.2-Exp thinking 0.52%
  45. 45 GLM-4.6 0.52%
  46. 46 Qwen3.5-9B 0.52%
  47. 47 Kimi K2 Thinking 0%
#ModelScoreSettingDateSource
1 Claude Opus 4.8 81.3% max May 2026 Official leaderboard
2 GPT-5.5 80.2% xhigh Apr 2026 Official leaderboard
3 GPT-5.4 Pro 69.8% xhigh Mar 2026 Official leaderboard
4 Kimi K3 65.6% thinking Jul 2026 Official leaderboard
5 Gemini 3.1 Pro 60.9% Feb 2026 Official leaderboard
6 GPT-5.4 54.2% xhigh Mar 2026 Official leaderboard
7 Claude Opus 4.7 40.6% xhigh Apr 2026 Official leaderboard
8 Claude Opus 4.6 34.5% high Feb 2026 Official leaderboard
9 Gemini 3.5 Flash 32.3% May 2026 Official leaderboard
10 DeepSeek-V4-Pro 28.1% max Apr 2026 Official leaderboard
11 DeepSeek-V4-Flash 27.1% max Apr 2026 Official leaderboard
12 Gemini 3.6 Flash 26% Jul 2026 Official leaderboard
13 Kimi K2.6 24% thinking Apr 2026 Official leaderboard
14 Gemini 3 Pro 23.4% Nov 2025 Official leaderboard
15 GLM-5.2 19.8% Jun 2026 Official leaderboard
16 Gemini 3 Flash 15.6% Dec 2025 Official leaderboard
17 Step 3.7 Flash 14.6% May 2026 Official leaderboard
18 GPT-5.2 13.5% high Dec 2025 Official leaderboard
19 Step 3.5 Flash 13.5% Feb 2026 Official leaderboard
20 GLM-5.1 11.5% Apr 2026 Official leaderboard
21 GLM-5 10.9% Feb 2026 Official leaderboard
22 DeepSeek-V3.2-Speciale 9.38% Dec 2025 Official leaderboard
23 Kimi K2.5 8.85% thinking Jan 2026 Official leaderboard
24 Nemotron 3 Super 7.81% Mar 2026 Official leaderboard
25 Qwen3-235B-A22B-Thinking Jul 2025 5.21% Jul 2025 Official leaderboard
26 Grok 4 Fast 5.21% Sep 2025 Official leaderboard
27 Grok 4.1 Fast 5.21% Nov 2025 Official leaderboard
28 Qwen3.5-35B-A3B 4.17% Feb 2026 Official leaderboard
29 Grok 4 2.08% Jul 2025 Official leaderboard
30 Qwen3-4B-Thinking-2507 2.08% Aug 2025 Official leaderboard
31 GPT-5 2.08% high, MathArena agent Aug 2025 Official leaderboard
32 DeepSeek-V3.2 2.08% thinking Dec 2025 Official leaderboard
33 Qwen3.5-27B 2.08% Feb 2026 Official leaderboard
34 Claude Sonnet 4.5 1.56% thinking Sep 2025 Official leaderboard
35 QED-Nano 1.56% Feb 2026 Official leaderboard
36 DeepSeek-R1 May 2025 1.04% May 2025 Official leaderboard
37 GLM-4.5 1.04% Aug 2025 Official leaderboard
38 gpt-oss-120b 1.04% high Aug 2025 Official leaderboard
39 GPT-5 mini 1.04% high Aug 2025 Official leaderboard
40 GPT-5.1 1.04% high Nov 2025 Official leaderboard
41 Gemini 2.5 Pro Jun 2025 0.52% Jun 2025 Official leaderboard
42 Qwen3-30B-A3B-Thinking Jul 2025 0.52% Jul 2025 Official leaderboard
43 DeepSeek-V3.1 0.52% thinking Aug 2025 Official leaderboard
44 DeepSeek-V3.2-Exp 0.52% thinking Sep 2025 Official leaderboard
45 GLM-4.6 0.52% Sep 2025 Official leaderboard
46 Qwen3.5-9B 0.52% Feb 2026 Official leaderboard
47 Kimi K2 Thinking 0% Nov 2025 Official leaderboard

Labs report scores for 13 models in their cards. We plot every source in the charts below.

The best score, 81.3%, costs $4.59 a task

Runs on the cost frontier score higher than any run that's cheaper. GPT-5.5 at xhigh comes within 1 points of the best for $1.42 a task, 3 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Qwen xAI DeepSeek StepFun Google DeepMind OpenAI Anthropic Moonshot AI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Opus 4.8, max Kimi K3, thinking Gemini 3.1 Pro Gemini 3.5 Flash DeepSeek-V4-Flash, max Step 3.5 Flash DeepSeek-V3.2-Speciale 0% 20% 40% 60% 80% 100% $0.001 $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Opus 4.8, max Kimi K3, thinking
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.8 max 81.3% $4.59 183,562
GPT-5.5 xhigh 80.2% $1.42 47,166
GPT-5.4 Pro xhigh 69.8% $8.65 46,663
Kimi K3 thinking 65.6% $1.11 74,265
Gemini 3.1 Pro 60.9% $0.41 33,915
GPT-5.4 xhigh 54.2% $1.03 67,637
Claude Opus 4.7 xhigh 40.6% $2.10 83,922
Claude Opus 4.6 high 34.5% $2.36 94,457
Gemini 3.5 Flash 32.3% $0.30 32,815
DeepSeek-V4-Pro max 28.1% $0.42 120,214
DeepSeek-V4-Flash max 27.1% $0.05 179,799
Gemini 3.6 Flash 26% $0.19 25,665
Kimi K2.6 thinking 24% $0.32 80,726
Gemini 3 Pro 23.4% $0.28 23,601
GLM-5.2 19.8% $0.18 43,072
Gemini 3 Flash 15.6% $0.10 34,852
Step 3.7 Flash 14.6% $0.07 248,920
GPT-5.2 high 13.5% $1.00 71,416
Step 3.5 Flash 13.5% $0.04 149,104
GLM-5.1 11.5% $0.27 85,816
GLM-5 10.9% $0.25 78,269
DeepSeek-V3.2-Speciale 9.38% $0.03 73,369
Kimi K2.5 thinking 8.85% $0.18 60,555
Nemotron 3 Super 7.81% 101,105
Qwen3-235B-A22B-Thinking Jul 2025 5.21% $0.05 42,861
Grok 4 Fast 5.21% $0.01 22,794
Grok 4.1 Fast 5.21% $0.01 26,208
Qwen3.5-35B-A3B 4.17% $0.09 65,584
Grok 4 2.08% $0.52 34,485
Qwen3-4B-Thinking-2507 2.08% $0.008 28,284
GPT-5 high MathArena agent 2.08% $3.83 376,117
DeepSeek-V3.2 thinking 2.08% $0.02 43,901
Qwen3.5-27B 2.08% $0.10 61,939
Claude Sonnet 4.5 thinking 1.56% $0.38 25,293
QED-Nano 1.56% 91,501
DeepSeek-R1 May 2025 1.04% $0.08 37,304
GLM-4.5 1.04% $0.08 34,285
gpt-oss-120b high 1.04% $0.03 45,365
GPT-5 high 1.04% $0.46 46,122
GPT-5 mini high 1.04% $0.07 34,849
GPT-5.1 high 1.04% $0.55 54,816
Gemini 2.5 Pro Jun 2025 0.52% $0.31 31,181
Qwen3-30B-A3B-Thinking Jul 2025 0.52% $0.009 30,727
DeepSeek-V3.1 thinking 0.52% $0.07 33,355
DeepSeek-V3.2-Exp thinking 0.52% $0.01 33,813
GLM-4.6 0.52% $0.07 31,749
Qwen3.5-9B 0.52% $0.009 59,074
Kimi K2 Thinking 0% $0.15 58,028

9 models have held number one since its release in Feb 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

DeepSeek xAI Qwen Google DeepMind Anthropic OpenAI
Own leaderboard Not tested since Jul 2026
Lab claims Passed 85% in Apr 2026, by DeepSeek-V4-Pro

The best score passed 85% in Apr 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) DeepSeek-V4-Pro GPT-5.5 GPT-5.4 Pro Gemini 3.1 Pro Claude Opus 4.6 Gemini 3 Pro Qwen3-235B-A22B-Thinking 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) DeepSeek-V4-Pro GPT-5.5

4 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
ByteDance Seed
DeepSeek
Qwen
NVIDIA
Tencent
DateLaunch postScoreAs the post writes it
17 Oct 2026 DeepSeek V4.1 Flash 1017 72.1 Apex Bench v1.1
15 Oct 2026 Nemotron 3.5 Super VL 88.4 APEX shortlist
10 Sep 2026 DeepSeek V4.1 Flash 65.6 MathArena Apex (Pass@1)
28 Aug 2026 Tencent Hy4 preview 74.2 MathArena Apex 2025
2 Jul 2026 Tencent Hy3 38.7 MathArena Apex
4 Jun 2026 Nemotron 3 Ultra 74.9 Apex-Shortlist (no tools); with tools 84.8
1 Jun 2026 Qwen3.7-Plus 22.7 Apex
20 May 2026 Qwen3.7-Max 44.5 Apex
24 Apr 2026 DeepSeek-V4-Pro 90.2 Apex Shortlist (Pass@1)
24 Apr 2026 DeepSeek-V4-Pro 38.3 Apex (Pass@1)
14 Feb 2026 Seed2.0 Pro 20.3 MathArenaApex 20.3 (shortlist 82.1)
18 Nov 2025 Gemini 3 Pro 23.4 MathArena Apex

7 system cards report it, from 3 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
15 Oct 2026 Nemotron 3.5 Super VL technical report 88.4 Apex-Shortlist, no tools, mean of 16 (Table 10) Capabilities
10 Sep 2026 DeepSeek V4.1 Flash technical report 65.6 MathArena Apex (Pass@1) (Table 3) Capabilities
23 Jun 2026 Seed2.1 Pro model card 31.3 MathArena Apex (Table 11) Capabilities
9 Jun 2026 Nemotron 3 Ultra technical report 74.9 Apex-Shortlist (no tools), 84.8 with tools (Table 10) Capabilities
26 Apr 2026 DeepSeek-V4-Pro technical report 38.3 Apex (Pass@1), DS-V4-Pro Max (Table 6) Capabilities
26 Apr 2026 DeepSeek-V4-Flash technical report 33 Apex (Pass@1), DeepSeek-V4-Flash Max (Table 7) Capabilities
14 Feb 2026 Seed2.0 Pro model card 20.3 MathArenaApex 20.3, MathArenaApex (shortlist) 82.1 (Table 3) Capabilities

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%
  • ArXivMath In use Final-answer research-level math problems extracted monthly from recent arXiv papers and filtered to be self-contained. GPT-6.1 Sol 94.2%