Benchmark Almanac

MATH-500

LongCat-Flash-Thinking 99.2% leads since Sep 2025 Fading

Models solve 500 competition math problems drawn from the MATH test set. Final answers are checked against the reference answers.

Near saturation, since DeepSeek-R1 already scored 97.3 percent in January 2025.

Made by OpenAI Released May 2023 Competition problems with answers All tasks public Last in a launch post Oct 2026 , dropped after it saturated Last model added Oct 2026
Level
Competition The strongest students, in olympiad-style contests
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
openai/prm800k 2,156 stars Paper Dataset
67
Models scored in lab cards
50
Launch posts list it, from 29 labs
43
System cards report it, from 18 labs

LongCat-Flash-Thinking leads with 99.2%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

67 models. Scroll sideways for the rest.

  1. 99.2 LongCat-Flash-Thinking
  2. 98.6 Sarvam 105B
  3. 98.6 Cogito v2.1
  4. 98.2 GLM-4.5
  5. 98.1 GLM-4.5-Air
  6. 98.1 INTELLECT-3
  7. 98.0 Qwen3-235B-A22B
  8. 98.0 Qwen3-30B-A3B
  9. 98.0 InternVL3.5-241B-A28B
  10. 97.9 o3-mini
  11. 97.8 nvidia-nemotron-nano-9b-v2
  12. 97.6 LongCat-Flash-Omni
  13. 97.4 Qwen3-8B
  14. 97.4 Kimi K2
  15. 97.4 Llama-3.3-Nemotron-Super-49B-v1.5
  16. 97.4 Falcon H1R 7B
  17. 97.3 DeepSeek-R1
  18. 97.2 Qwen3-32B
  19. 97.0 Llama-3.1-Nemotron-Ultra-253B-v1
  20. 97.0 Qwen3-4B
  21. 97.0 Sarvam 30B
  22. 96.9 Nemotron Nano 12B v2 VL
  23. 96.8 Qwen3-14B
  24. 96.8 MiniMax-M1-80k
  25. 96.6 Llama-3.3-Nemotron-Super-49B-v1
  26. 96.6 Code World Model (CWM)
  27. 96.4 o1
  28. 96.4 ERNIE-4.5-300B-A47B
  29. 96.4 LongCat-Flash-Chat
  30. 96.3 Hermes 4 405B
  31. 96.2 Kimi k1.5
  32. 96.2 Claude 3.7 Sonnet
  33. 96.2 Hunyuan-T1
  34. 96.0 LFM2.5-24B-A2B
  35. 95.9 Magistral Small 1.0
  36. 95.8 MiMo-7B-RL
  37. 95.7 EXAONE Deep 32B
  38. 95.4 Solar Pro 2
  39. 95.4 Nemotron 3 Nano 4B
  40. 95.0 Llama 4 Behemoth
  41. 94.6 Phi-4-mini-reasoning
  42. 94.5 DeepSeek-R1-Distill-Llama-70B
  43. 94.3 DeepSeek-R1-Distill-Qwen-32B
  44. 94.3 Magistral Medium 1.0
  45. 93.9 DeepSeek-R1-Distill-Qwen-14B
  46. 93.8 Hermes 4.3 36B
  47. 93.4 Qwen3-1.7B
  48. 92.5 Phi-4-mini-flash-reasoning
  49. 92.1 Trinity Mini
  50. 91.0 Mistral Medium 3
  51. 90.6 Reka Flash 3
  52. 90.2 DeepSeek-V3
  53. 88.8 LFM2.5-8B-A1B
  54. 88.0 LFM2.5-1.2B-Thinking
  55. 85.5 o1-preview
  56. 83.9 DeepSeek-R1-Distill-Qwen-1.5B
  57. 83.8 Falcon-H1-34B
  58. 83.0 InternLM3-8B-Instruct
  59. 83.0 Mercury
  60. 82.0 Amazon Nova Premier
  61. 74.2 LFM2-8B-A1B
  62. 71.8 Phi-4 Mini
  63. 69.0 Granite 3.3 8B Instruct
  64. 43.4 BitNet b1.58 2B4T
  65. 42.4 LFM2-1.2B
  66. 18.4 Qwen-14B
  67. 6.80 Qwen-7B
  1. 1 LongCat-Flash-Thinking 99.2%
  2. 2 Sarvam 105B 98.6%
  3. 3 Cogito v2.1 98.6%
  4. 4 GLM-4.5 98.2%
  5. 5 GLM-4.5-Air 98.1%
  6. 6 INTELLECT-3 98.1%
  7. 7 Qwen3-235B-A22B 98%
  8. 8 Qwen3-30B-A3B 98%
  9. 9 InternVL3.5-241B-A28B 98%
  10. 10 o3-mini 97.9%
  11. 11 nvidia-nemotron-nano-9b-v2 97.8%
  12. 12 LongCat-Flash-Omni 97.6%
  13. 13 Qwen3-8B 97.4%
  14. 14 Kimi K2 Jul 2025 97.4%
  15. 15 Llama-3.3-Nemotron-Super-49B-v1.5 97.4%
  16. 16 Falcon H1R 7B 97.4%
  17. 17 DeepSeek-R1 97.3%
  18. 18 Qwen3-32B 97.2%
  19. 19 Llama-3.1-Nemotron-Ultra-253B-v1 97%
  20. 20 Qwen3-4B 97%
  21. 21 Sarvam 30B 97%
  22. 22 Nemotron Nano 12B v2 VL 96.9%
  23. 23 Qwen3-14B 96.8%
  24. 24 MiniMax-M1-80k 96.8%
  25. 25 Llama-3.3-Nemotron-Super-49B-v1 96.6%
  26. 26 Code World Model (CWM) 96.6%
  27. 27 o1 96.4%
  28. 28 ERNIE-4.5-300B-A47B 96.4%
  29. 29 LongCat-Flash-Chat 96.4%
  30. 30 Hermes 4 405B 96.3%
  31. 31 Kimi k1.5 96.2%
  32. 32 Claude 3.7 Sonnet 96.2%
  33. 33 Hunyuan-T1 96.2%
  34. 34 LFM2.5-24B-A2B 96%
  35. 35 Magistral Small 1.0 95.9%
  36. 36 MiMo-7B-RL 95.8%
  37. 37 EXAONE Deep 32B 95.7%
  38. 38 Solar Pro 2 95.4%
  39. 39 Nemotron 3 Nano 4B 95.4%
  40. 40 Llama 4 Behemoth 95%
  41. 41 Phi-4-mini-reasoning 94.6%
  42. 42 DeepSeek-R1-Distill-Llama-70B 94.5%
  43. 43 DeepSeek-R1-Distill-Qwen-32B 94.3%
  44. 44 Magistral Medium 1.0 94.3%
  45. 45 DeepSeek-R1-Distill-Qwen-14B 93.9%
  46. 46 Hermes 4.3 36B 93.8%
  47. 47 Qwen3-1.7B 93.4%
  48. 48 Phi-4-mini-flash-reasoning 92.5%
  49. 49 Trinity Mini 92.1%
  50. 50 Mistral Medium 3 91%
  51. 51 Reka Flash 3 90.6%
  52. 52 DeepSeek-V3 90.2%
  53. 53 LFM2.5-8B-A1B 88.8%
  54. 54 LFM2.5-1.2B-Thinking 88%
  55. 55 o1-preview 85.5%
  56. 56 DeepSeek-R1-Distill-Qwen-1.5B 83.9%
  57. 57 Falcon-H1-34B 83.8%
  58. 58 InternLM3-8B-Instruct 83%
  59. 59 Mercury 83%
  60. 60 Amazon Nova Premier 82%
  61. 61 LFM2-8B-A1B 74.2%
  62. 62 Phi-4 Mini 71.8%
  63. 63 Granite 3.3 8B Instruct 69%
  64. 64 BitNet b1.58 2B4T 43.4%
  65. 65 LFM2-1.2B 42.4%
  66. 66 Qwen-14B 18.4%
  67. 67 Qwen-7B 6.80%
#ModelScoreSettingDateSource
1 LongCat-Flash-Thinking 99.2% Sep 2025 Model card
2 Sarvam 105B 98.6% Mar 2026 Model card
3 Cogito v2.1 98.6% Nov 2025 Model card
4 GLM-4.5 98.2% Jul 2025 Model card
5 GLM-4.5-Air 98.1% Aug 2025 Model card
6 INTELLECT-3 98.1% Nov 2025 Model card
7 Qwen3-235B-A22B 98% Apr 2025 Model card
8 Qwen3-30B-A3B 98% May 2025 Model card
9 InternVL3.5-241B-A28B 98% Aug 2025 Model card
10 o3-mini 97.9% Jan 2025 Model card
11 nvidia-nemotron-nano-9b-v2 97.8% Aug 2025 Model card
12 LongCat-Flash-Omni 97.6% Oct 2025 Model card
13 Qwen3-8B 97.4% May 2025 Model card
14 Kimi K2 Jul 2025 97.4% Jul 2025 Model card
15 Llama-3.3-Nemotron-Super-49B-v1.5 97.4% Jul 2025 Model card
16 Falcon H1R 7B 97.4% Jan 2026 Model card
17 DeepSeek-R1 97.3% Jan 2025 Model card
18 Qwen3-32B 97.2% May 2025 Model card
19 Llama-3.1-Nemotron-Ultra-253B-v1 97% Apr 2025 Model card
20 Qwen3-4B 97% May 2025 Model card
21 Sarvam 30B 97% Mar 2026 Model card
22 Nemotron Nano 12B v2 VL 96.9% Nov 2025 Model card
23 Qwen3-14B 96.8% May 2025 Model card
24 MiniMax-M1-80k 96.8% Jun 2025 Model card
25 Llama-3.3-Nemotron-Super-49B-v1 96.6% Mar 2025 Model card
26 Code World Model (CWM) 96.6% Sep 2025 Model card
27 o1 96.4% Dec 2024 Model card
28 ERNIE-4.5-300B-A47B 96.4% Jun 2025 Model card
29 LongCat-Flash-Chat 96.4% Aug 2025 Model card
30 Hermes 4 405B 96.3% Aug 2025 Model card
31 Kimi k1.5 96.2% Jan 2025 Model card
32 Claude 3.7 Sonnet 96.2% Feb 2025 Model card
33 Hunyuan-T1 96.2% Mar 2025 Model card
34 LFM2.5-24B-A2B 96% Oct 2026 Model card
35 Magistral Small 1.0 95.9% Jun 2025 Model card
36 MiMo-7B-RL 95.8% Apr 2025 Model card
37 EXAONE Deep 32B 95.7% Mar 2025 Model card
38 Solar Pro 2 95.4% Jul 2025 Model card
39 Nemotron 3 Nano 4B 95.4% Mar 2026 Model card
40 Llama 4 Behemoth 95% Apr 2025 Model card
41 Phi-4-mini-reasoning 94.6% Apr 2025 Model card
42 DeepSeek-R1-Distill-Llama-70B 94.5% Jan 2025 Model card
43 DeepSeek-R1-Distill-Qwen-32B 94.3% Jan 2025 Model card
44 Magistral Medium 1.0 94.3% Jun 2025 Model card
45 DeepSeek-R1-Distill-Qwen-14B 93.9% Jan 2025 Model card
46 Hermes 4.3 36B 93.8% Dec 2025 Model card
47 Qwen3-1.7B 93.4% May 2025 Model card
48 Phi-4-mini-flash-reasoning 92.5% Jul 2025 Model card
49 Trinity Mini 92.1% Dec 2025 Model card
50 Mistral Medium 3 91% May 2025 Model card
51 Reka Flash 3 90.6% Mar 2025 Model card
52 DeepSeek-V3 90.2% Dec 2024 Model card
53 LFM2.5-8B-A1B 88.8% May 2026 Model card
54 LFM2.5-1.2B-Thinking 88% Jan 2026 Model card
55 o1-preview 85.5% Sep 2024 Model card
56 DeepSeek-R1-Distill-Qwen-1.5B 83.9% Jan 2025 Model card
57 Falcon-H1-34B 83.8% May 2025 Model card
58 InternLM3-8B-Instruct 83% Jan 2025 Model card
59 Mercury 83% Jun 2025 Model card
60 Amazon Nova Premier 82% Apr 2025 Model card
61 LFM2-8B-A1B 74.2% Oct 2025 Model card
62 Phi-4 Mini 71.8% Mar 2025 Model card
63 Granite 3.3 8B Instruct 69% Apr 2025 Model card
64 BitNet b1.58 2B4T 43.4% Apr 2025 Model card
65 LFM2-1.2B 42.4% Nov 2025 Model card
66 Qwen-14B 18.4% Sep 2023 Model card
67 Qwen-7B 6.80% Sep 2023 Model card

8 models have held number one since its release in May 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Qwen OpenAI DeepSeek Z.ai Meituan
Lab claims Passed 85% in Sep 2024, by o1-preview
LongCat-Flash-Thinking 99.2%

The best score passed 85% in Sep 2024

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) LongCat-Flash-Thinking o3-mini o1-preview Qwen-14B 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) LongCat-Flash-Thinking o1-preview

Labs headline it less often now, last in Oct 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
DeepSeek
Shanghai AI Lab
Moonshot AI
Anthropic
Reka
Baidu
NVIDIA
LG AI Research
Tencent
Meta
Microsoft
IBM
Qwen
Xiaomi Corp
Amazon
Mistral AI
Technology Innovation Institute
MiniMax
Inception Labs
Upstage
Z.ai
Nous Research
Meituan
Liquid AI
Deep Cogito
Prime Intellect
Arcee AI
Sarvam AI
DateLaunch postScoreAs the post writes it
13 Oct 2026 LFM2.5-24B-A2B 96 MATH-500
28 May 2026 LFM2.5-8B-A1B 88.8 MATH500
16 Mar 2026 Nemotron 3 Nano 4B 95.4 MATH500 (reasoning on)
6 Mar 2026 Sarvam 30B 97 Math500
6 Mar 2026 Sarvam 105B 98.6 Math500
20 Jan 2026 LFM2.5-1.2B-Thinking 88 MATH-500 (± 0.72)
3 Dec 2025 Hermes 4.3 36B 93.8 MATH-500
1 Dec 2025 Trinity Mini 92.1 Math-500
26 Nov 2025 INTELLECT-3 98.1 MATH-500
19 Nov 2025 Cogito v2.1 98.6 MATH-500
31 Oct 2025 LongCat-Flash-Omni 97.6 MATH500(acc)
7 Oct 2025 LFM2-8B-A1B 74.2 MATH 500
24 Sep 2025 Code World Model (CWM) 96.6 Math-500
21 Sep 2025 LongCat-Flash-Thinking 99.2 MATH500(Mean@1)
29 Aug 2025 LongCat-Flash-Chat 96.4 MATH500(acc)
26 Aug 2025 Hermes 4 405B 96.3 MATH-500 (reasoning)
25 Aug 2025 InternVL3.5-241B-A28B 98 MATH500
18 Aug 2025 nvidia-nemotron-nano-9b-v2 97.8 MATH500
28 Jul 2025 GLM-4.5 98.2 MATH 500
25 Jul 2025 Llama-3.3-Nemotron-Super-49B-v1.5 97.4 MATH500 (pass@1, avg of 4)
11 Jul 2025 Kimi K2 97.4 MATH-500, Acc
10 Jul 2025 Solar Pro 2 95.4 Math500
9 Jul 2025 Phi-4-mini-flash-reasoning 92.5 Math500
30 Jun 2025 ERNIE-4.5-300B-A47B 96.4 MATH-500
26 Jun 2025 Mercury 83 MATH-500 (% accuracy)
16 Jun 2025 MiniMax-M1-80k 96.8 MATH-500
20 May 2025 Falcon-H1-34B 83.8 MATH-500
7 May 2025 Mistral Medium 3 91 Math500 Instruct 0-shot
30 Apr 2025 Phi-4-mini-reasoning 94.6 MATH-500
30 Apr 2025 Amazon Nova Premier 82 MATH-500
29 Apr 2025 MiMo-7B-RL 95.8 MATH-500 (Pass@1)
29 Apr 2025 Qwen3-235B-A22B 98 MATH-500
16 Apr 2025 Granite 3.3 8B Instruct 69 MATH-500
15 Apr 2025 BitNet b1.58 2B4T 43.4 MATH-500
7 Apr 2025 Llama-3.1-Nemotron-Ultra-253B-v1 97 MATH500 (reasoning on)
5 Apr 2025 Llama 4 Behemoth 95 MATH-500
24 Mar 2025 DeepSeek-V3 MATH-500 (Pass@1)
21 Mar 2025 Hunyuan-T1 96.2 MATH-500
18 Mar 2025 EXAONE Deep 32B 95.7 MATH-500 (pass@1)
18 Mar 2025 Llama-3.3-Nemotron-Super-49B-v1 96.6 MATH500 (reasoning on)
16 Mar 2025 ERNIE 4.5 Math-500
10 Mar 2025 Reka Flash 3 90.6 MATH-500
24 Feb 2025 Claude 3.7 Sonnet 96.2 Math problem-solving MATH 500 (64K extended thinking)
31 Jan 2025 o3-mini 97.9 Math (pass@1)
20 Jan 2025 Kimi k1.5 96.2 MATH-500 (EM), long-CoT
20 Jan 2025 DeepSeek-R1 97.3 MATH-500 (Pass@1)
13 Jan 2025 InternLM3-8B-Instruct 83 MATH-500(0-shot), Thinking Mode
26 Dec 2024 DeepSeek-V3 90.2 MATH-500 (EM)
17 Dec 2024 o1 96.4 MATH (pass @1)
12 Sep 2024 o1-preview 85.5 MATH pass@1 (500-problem split)

43 system cards report it, from 18 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
5 Jan 2026 Falcon H1R 7B technical report 97.4 MATH500 pass@1 (Table 4) Capabilities
28 Nov 2025 LFM2-8B-A1B technical report 74.2 MATH 500, LFM2-8B-A1B (Table 7) Capabilities
28 Nov 2025 LFM2-1.2B technical report 42.4 MATH 500, LFM2-1.2B (Table 6) Capabilities
6 Nov 2025 Nemotron Nano 12B v2 VL technical report 96.9 MATH-500, reasoning on, SFT stage 4 final model, mean of 4 runs (Table 6) Capabilities
31 Oct 2025 LongCat-Flash-Omni technical report 97.6 MATH500 (acc) (Table 13) Capabilities
30 Sep 2025 Code World Model (CWM) technical report 96.6 Math-500, pass@1 averaged over 20 samples (Table 12) Capabilities
23 Sep 2025 LongCat-Flash-Thinking technical report 99.2 MATH-500(Mean@1) (Table 2) Capabilities
1 Sep 2025 LongCat-Flash-Chat technical report 96.4 MATH500 (acc) (Table 3) Capabilities
25 Aug 2025 InternVL3.5-241B-A28B technical report 98 MATH500 (Table 2) Capabilities
25 Aug 2025 Hermes 4 405B technical report 96.2 MATH-500, reasoning mode (Table 3). Non-reasoning 73.8. v1 printed 96.3 Capabilities
8 Aug 2025 GLM-4.5 technical report 98.2 MATH 500 (Table 4) Capabilities
8 Aug 2025 GLM-4.5-Air technical report 98.1 MATH 500 (Table 4) Capabilities
30 Jul 2025 Falcon-H1-34B technical report 83.8 MATH-500 (Table 24) Capabilities
21 Jul 2025 Kimi K2 technical report 97.4 MATH-500 (Acc.) (Table 3) Capabilities
9 Jul 2025 Phi-4-mini-flash-reasoning technical report 92.5 Math500, pass@1 averaged over 8 samples (Table 4) Capabilities
29 Jun 2025 ERNIE-4.5-300B-A47B technical report 96.4 MATH-500 (Table 5) Capabilities
16 Jun 2025 MiniMax-M1-80k technical report 96.8 MATH-500 (Table 2) Capabilities
12 Jun 2025 Magistral Small 1.0 technical report 95.9 MATH-500, SFT + RL (Magistral Small) column (Table 3) Capabilities
12 Jun 2025 Magistral Medium 1.0 technical report 94.3 MATH-500 (Table 2) Capabilities
14 May 2025 Qwen3-8B technical report 97.4 MATH-500, thinking mode (Table 17). Non-thinking 87.4 (Table 18) Capabilities
14 May 2025 Qwen3-4B technical report 97 MATH-500, thinking mode (Table 17). Non-thinking 84.8 (Table 18) Capabilities
14 May 2025 Qwen3-32B technical report 97.2 MATH-500, thinking mode (Table 13). Non-thinking 88.6 (Table 14) Capabilities
14 May 2025 Qwen3-30B-A3B technical report 98 MATH-500, thinking mode (Table 15). Non-thinking 89.8 (Table 16) Capabilities
14 May 2025 Qwen3-235B-A22B technical report 98 MATH-500, thinking mode (Table 11). Non-thinking 91.2 (Table 12) Capabilities
14 May 2025 Qwen3-14B technical report 96.8 MATH-500, thinking mode (Table 15). Non-thinking 90.0 (Table 16) Capabilities
14 May 2025 Qwen3-1.7B technical report 93.4 MATH-500, thinking mode (Table 19). Non-thinking 73.0 (Table 20) Capabilities
12 May 2025 MiMo-7B-RL technical report 95.8 MATH500 (Pass@1) (Table 4, Table 5, Table 6) Capabilities
2 May 2025 Llama-3.3-Nemotron-Super-49B-v1 technical report 96.6 MATH500, reasoning on, reasoning off 74.0 (Table 4) Capabilities
2 May 2025 Llama-3.1-Nemotron-Ultra-253B-v1 technical report 97 MATH500, reasoning on, reasoning off 80.4 (Table 5) Capabilities
30 Apr 2025 Phi-4-mini-reasoning technical report 94.6 MATH-500, pass@1 (Table 3) Capabilities
30 Apr 2025 Amazon Nova Premier technical report 82 MATH-500, 0-shot CoT, +/-3.4 (Table 1) Capabilities
16 Apr 2025 BitNet b1.58 2B4T technical report 43.4 MATH-500 (0-shot; EM) (Table 1) Capabilities
16 Mar 2025 EXAONE Deep 32B technical report 95.7 MATH-500, pass@1 with k=8 (Table 2) Capabilities
3 Mar 2025 Phi-4 Mini technical report 71.8 MATH-500, Phi-4-Mini row of the CoT reasoning comparison (Table 9) Capabilities
20 Jan 2025 Kimi k1.5 technical report 96.2 MATH-500 (EM), long-CoT (Table 2). Short-CoT 94.6 (Table 3) Capabilities
20 Jan 2025 DeepSeek-R1 technical report 97.3 MATH-500 (Pass@1), DeepSeek-R1 (Table 8, same as final column of Table 3) Capabilities
20 Jan 2025 DeepSeek-R1-Distill-Qwen-32B technical report 94.3 MATH-500 pass@1, DeepSeek-R1-Distill-Qwen-32B (Table 15) Capabilities
20 Jan 2025 DeepSeek-R1-Distill-Qwen-14B technical report 93.9 MATH-500 pass@1, DeepSeek-R1-Distill-Qwen-14B (Table 15) Capabilities
20 Jan 2025 DeepSeek-R1-Distill-Qwen-1.5B technical report 83.9 MATH-500 pass@1, DeepSeek-R1-Distill-Qwen-1.5B (Table 15) Capabilities
20 Jan 2025 DeepSeek-R1-Distill-Llama-70B technical report 94.5 MATH-500 pass@1, DeepSeek-R1-Distill-Llama-70B (Table 15) Capabilities
26 Dec 2024 DeepSeek-V3 technical report 90.2 MATH-500 (EM), DeepSeek-V3 (Table 6) Capabilities
28 Sep 2023 Qwen-7B technical report 6.8 MATH on the Lightman et al. (2023) test set, greedy decoding (Table 12) Capabilities
28 Sep 2023 Qwen-14B technical report 18.4 MATH on the Lightman et al. (2023) test set, greedy decoding (Table 12) Capabilities

Versions

Replaces MATH Replaced by AIME 2025

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%