Benchmark Almanac

PolyMath

Qwen3.7-Max 86.5% leads since May 2026 In use

Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked.

Some model cards report only the English subset, PolyMath-en.

In the authors’ words

In this paper, we introduce PolyMath, a multilingual mathematical reasoning benchmark covering 18 languages and 4 easy-to-hard difficulty levels. Our benchmark ensures difficulty comprehensiveness, language diversity, and high-quality translation, making it a highly discriminative multilingual mathematical benchmark in the era of reasoning LLMs. We conduct a comprehensive evaluation for advanced LLMs and find that even Qwen-3-235B-A22B-Thinking and Gemini-2.5-pro, achieve only 54.6 and 52.2 benchmark scores, with about 40% accuracy under the highest level From a language perspective, our benchmark reveals several key challenges of LLMs in multilingual reasoning: (1) Reasoning performance varies widely across languages for current LLMs; (2) Input-output language consistency is low in reasoning LLMs and may be correlated with performance; (3) The thinking length differs significantly by language for current LLMs. Additionally, we demonstrate that controlling the output language in the instructions has the potential to affect reasoning performance, especially for some low-resource languages, suggesting a promising direction for improving multilingual capabilities in LLMs.

Abstract of PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts (arXiv:2504.18428, CC0)
Made by Qwen Team , Alibaba Group Released Apr 2025 Competition problems with answers All tasks public Last in a launch post Jul 2026 Last model added Aug 2026 Scores refreshed 2 Nov 2026
Level
Competition The strongest students, in olympiad-style contests
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
QwenLM/PolyMath 43 stars Paper Dataset Leaderboard
18
Models scored in lab cards
11
Launch posts list it, from 3 labs
11
System cards report it, from 3 labs

Qwen3.7-Max leads with 86.5% among models released by Jul 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in Jul 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

18 models. Scroll sideways for the rest.

  1. 86.5 Qwen3.7-Max
  2. 84.0 Qwen3.7-Plus
  3. 77.4 Qwen 3.6 Plus
  4. 73.3 Qwen3.5 397B-A17B
  5. 71.3 K-EXAONE 2.0
  6. 68.9 Qwen3.5-122B-A10B
  7. 65.1 Kimi K2
  8. 60.1 Qwen3-235B-A22B-Thinking
  9. 57.8 Qwen3-VL-235B-A22B
  10. 56.3 Qwen3-Next-80B-A3B
  11. 54.7 Qwen3-235B-A22B
  12. 47.4 Qwen3-32B
  13. 46.1 Qwen3-30B-A3B
  14. 45.8 Qwen3-14B
  15. 42.7 Qwen3-8B
  16. 40.0 Qwen3-4B
  17. 37.9 Qwen3-Omni-30B-A3B
  18. 25.2 Qwen3-1.7B
  1. 1 Qwen3.7-Max 86.5%
  2. 2 Qwen3.7-Plus 84%
  3. 3 Qwen 3.6 Plus 77.4%
  4. 4 Qwen3.5 397B-A17B 73.3%
  5. 5 K-EXAONE 2.0 71.3%
  6. 6 Qwen3.5-122B-A10B 68.9%
  7. 7 Kimi K2 Jul 2025 65.1%
  8. 8 Qwen3-235B-A22B-Thinking Jul 2025 60.1%
  9. 9 Qwen3-VL-235B-A22B 57.8%
  10. 10 Qwen3-Next-80B-A3B 56.3%
  11. 11 Qwen3-235B-A22B 54.7%
  12. 12 Qwen3-32B 47.4%
  13. 13 Qwen3-30B-A3B 46.1%
  14. 14 Qwen3-14B 45.8%
  15. 15 Qwen3-8B 42.7%
  16. 16 Qwen3-4B 40%
  17. 17 Qwen3-Omni-30B-A3B 37.9%
  18. 18 Qwen3-1.7B 25.2%
#ModelScoreSettingDateSource
1 Qwen3.7-Max 86.5% May 2026 Model card
2 Qwen3.7-Plus 84% Jun 2026 Model card
3 Qwen 3.6 Plus 77.4% Apr 2026 Model card
4 Qwen3.5 397B-A17B 73.3% Feb 2026 Model card
5 K-EXAONE 2.0 71.3% Jul 2026 Model card
6 Qwen3.5-122B-A10B 68.9% Feb 2026 Model card
7 Kimi K2 Jul 2025 65.1% Jul 2025 Model card
8 Qwen3-235B-A22B-Thinking Jul 2025 60.1% Jul 2025 Model card
9 Qwen3-VL-235B-A22B 57.8% Nov 2025 Model card
10 Qwen3-Next-80B-A3B 56.3% Sep 2025 Model card
11 Qwen3-235B-A22B 54.7% Apr 2025 Model card
12 Qwen3-32B 47.4% May 2025 Model card
13 Qwen3-30B-A3B 46.1% May 2025 Model card
14 Qwen3-14B 45.8% May 2025 Model card
15 Qwen3-8B 42.7% May 2025 Model card
16 Qwen3-4B 40% May 2025 Model card
17 Qwen3-Omni-30B-A3B 37.9% Sep 2025 Model card
18 Qwen3-1.7B 25.2% May 2025 Model card

The benchmark's own leaderboard has 38 models. We plot every source in the charts below.

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Qwen3-235B-A22B
non-thinking thinking
54.6%
Qwen3-32B
non-thinking thinking
47.4%
Qwen3-30B-A3B
non-thinking thinking
46.1%
Qwen3-14B
non-thinking thinking
45.8%
Qwen3-8B
non-thinking thinking
42.7%
Claude 3.7 Sonnet
thinking
33.5%
ModelEffortHarnessScoreCost per taskOutput tokens
Qwen3-235B-A22B thinking 54.6%
Gemini 2.5 Pro Mar 2025 52.2%
Qwen3-32B thinking 47.4%
Seed-Thinking-v1.5 47.2%
DeepSeek-R1 47%
Qwen3-30B-A3B thinking 46.1%
QwQ-32B 45.9%
Qwen3-14B thinking 45.8%
o4-mini medium 45.6%
Qwen3-8B thinking 42.7%
Qwen3-4B thinking 40%
o1 38.9%
o3-mini medium 38.6%
Gemini 2.0 Flash Thinking Jan 2025 37.1%
o1-mini 36.6%
Grok 3 35.2%
DeepSeek-R1-Distill-Qwen-32B 35.1%
Claude 3.7 Sonnet thinking 33.5%
Grok-3 mini 32.3%
DeepSeek-V3 Mar 2025 30.7%
DeepSeek-R1-Distill-Llama-70B 29.4%
GPT-4.1 mini 28.8%
DeepSeek-R1-Distill-Qwen-14B 28.6%
DeepSeek-R1-Distill-Qwen-7B 28.3%
Qwen3-235B-A22B non-thinking 27%
GPT-4.5 26.9%
GPT-4.1 26.4%
Llama 4 Maverick 26.1%
Qwen3-1.7B thinking 25.2%
GPT-4o Mar 2025 24.3%
Qwen3-30B-A3B non-thinking 22.9%
Qwen3-32B non-thinking 22.5%
Qwen3-14B non-thinking 22%
Qwen2.5-Math-72B-Instruct 21%
Llama 4 Scout 20.9%
DeepSeek-V3 20.4%
Gemma 3 27B 20.3%
Claude 3.7 Sonnet 19.7%
Qwen2.5-Max 19.4%
Qwen3-8B non-thinking 18.8%
Phi-4 17.4%
Qwen2.5-72B 16.9%
GPT-4o Nov 2024 13.7%
Llama 3.3 70B Instruct 11.5%

2 models have held number one since its release in Apr 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Google DeepMind Qwen Moonshot AI
Own leaderboard Not tested since Apr 2025
Qwen3-235B-A22B 54.6%
Lab claims Passed 85% in May 2026, by Qwen3.7-Max
Qwen3.7-Max 86.5%

The best score passed 85% in May 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Qwen3.7-Max Qwen 3.6 Plus Kimi K2 Qwen3-235B-A22B DeepSeek-R1 o1 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Qwen3.7-Max Qwen 3.6 Plus

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Qwen
Moonshot AI
LG AI Research
DateLaunch postScoreAs the post writes it
31 Jul 2026 K-EXAONE 2.0 71.3 PolyMath (nine non-English supported languages)
1 Jun 2026 Qwen3.7-Plus 84 PolyMATH
20 May 2026 Qwen3.7-Max 86.5 PolyMATH
2 Apr 2026 Qwen 3.6 Plus 77.4 PolyMATH
24 Feb 2026 Qwen3.5-122B-A10B 68.9 PolyMATH
16 Feb 2026 Qwen3.5 397B-A17B 73.3 PolyMATH
22 Sep 2025 Qwen3-Omni-30B-A3B 37.9 PolyMATH
11 Sep 2025 Qwen3-Next-80B-A3B 56.3 PolyMATH
25 Jul 2025 Qwen3-235B-A22B-Thinking 60.1 PolyMATH
11 Jul 2025 Kimi K2 65.1 PolyMath-en, Avg@4
29 Apr 2025 Qwen3-235B-A22B 54.7 PolyMath

11 system cards report it, from 3 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
5 Aug 2026 K-EXAONE 2.0 technical report 71.3 PolyMath, nine non-English languages (per-language values in Table 7) (Table 6, Reasoning mode) Capabilities
26 Nov 2025 Qwen3-VL-235B-A22B technical report 57.8 PolyMATH (Table 6) Capabilities
22 Sep 2025 Qwen3-Omni-30B-A3B technical report 37.9 PolyMATH, Instruct (Table 4). Thinking 47.1 (Table 5) Capabilities
21 Jul 2025 Kimi K2 technical report 65.1 PolyMath-en (Avg@4) (Table 3) Capabilities
14 May 2025 Qwen3-8B technical report 42.7 PolyMath, thinking mode (Table 17). Non-thinking 18.8 (Table 18) Capabilities
14 May 2025 Qwen3-4B technical report 40 PolyMath, thinking mode (Table 17). Non-thinking 16.6 (Table 18) Capabilities
14 May 2025 Qwen3-32B technical report 47.4 PolyMath, thinking mode (Table 13). Non-thinking 22.5 (Table 14) Capabilities
14 May 2025 Qwen3-30B-A3B technical report 46.1 PolyMath, thinking mode (Table 15). Non-thinking 23.3 (Table 16) Capabilities
14 May 2025 Qwen3-235B-A22B technical report 54.7 PolyMath, thinking mode (Table 11). Non-thinking 27.0 (Table 12) Capabilities
14 May 2025 Qwen3-14B technical report 45.8 PolyMath, thinking mode (Table 15). Non-thinking 22.0 (Table 16) Capabilities
14 May 2025 Qwen3-1.7B technical report 25.2 PolyMath, thinking mode (Table 19). Non-thinking 10.3 (Table 20) Capabilities

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%
  • ArXivMath In use Final-answer research-level math problems extracted monthly from recent arXiv papers and filtered to be self-contained. GPT-6.1 Sol 94.2%