Benchmark Almanac

ArXivMath

GPT-6.1 Sol 94.2% leads since Sep 2026 In use

Final-answer research-level math problems extracted monthly from recent arXiv papers and filtered to be self-contained.

MathArena adds a new problem set each month from that month's arXiv papers. From the August 2026 set, models run in their own harnesses with Python and SageMath, and problems center on resolved conjectures. Scores before and after that change do not compare.

In the authors’ words

Large language models (LLMs) are becoming increasingly capable mathematical collaborators, but static benchmarks are no longer sufficient for evaluating progress: they are often narrow in scope, quickly saturated, and rarely updated. This makes it hard to compare models reliably and track progress over time. Instead, we need evaluation platforms: continuously maintained systems that run, aggregate, and analyze evaluations across many benchmarks to give a comprehensive picture of model performance within a broad domain. In this work, we build on the original MathArena benchmark by substantially broadening its scope from final-answer olympiad problems to a continuously maintained evaluation platform for mathematical reasoning with LLMs. MathArena now covers a much wider range of tasks, including proof-based competitions, research-level arXiv problems, and formal proof generation in Lean. Additionally, we maintain a clear evaluation protocol for all models and regularly design new benchmarks as model capabilities improve to ensure that MathArena remains challenging. Notably, the strongest model, GPT-5.5, now reaches 98% on the 2026 USA Math Olympiad and 74% on research-level questions, showing that frontier models can now comfortably solve extremely challenging mathematical problems. This highlights the importance of continuously maintained evaluation platforms like MathArena to track the rapid progress of LLMs in mathematical reasoning.

Abstract of Beyond Benchmarks: MathArena as an Evaluation Platform for Mathematics with LLMs (arXiv:2605.00674, CC0)
Made by MathArena (ETH Zurich SRI Lab) Released Feb 2026 Research mathematics All tasks public Last in a launch post Aug 2026 Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Research frontier Researchers, on open problems or tasks that take days
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
6
Models scored on its own leaderboard
3
Launch posts list it, from 2 labs
5
System cards report it, from 1 lab

GPT-6.1 Sol leads with 94.2%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.

  1. 1 GPT-6.1 Sol max 94.2%
  2. 2 Claude Opus 5.5 high 83.8%
  3. 3 Grok 4.7 xhigh 54%
  4. 4 Gemini 3.8 Flash 52.6%
  5. 5 DeepSeek V4.1 Flash max 47.4%
  6. 6 Muse Spark Open 43.5%
#ModelScoreSettingDateSource
1 GPT-6.1 Sol 94.2% max Sep 2026 Official leaderboard
2 Claude Opus 5.5 83.8% high Sep 2026 Official leaderboard
3 Grok 4.7 54% xhigh Sep 2026 Official leaderboard
4 Gemini 3.8 Flash 52.6% Sep 2026 Official leaderboard
5 DeepSeek V4.1 Flash 47.4% max Sep 2026 Official leaderboard
6 Muse Spark Open 43.5% Oct 2026 Official leaderboard

Labs report scores for 8 models in their cards. We plot every source in the charts below.

4 models have held number one since its release in Feb 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Google DeepMind xAI Anthropic OpenAI
Own leaderboard Passed 85% in Sep 2026, by GPT-6.1 Sol
GPT-6.1 Sol 94.2%
Lab claims Passed 85% in Jul 2026, by Claude Opus 5

The best score passed 85% in Jul 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-6.1 Sol Claude Opus 5 Claude Fable 5 Claude Opus 4.8 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-6.1 Sol Claude Fable 5

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Anthropic
Tencent
DateLaunch postScoreAs the post writes it
28 Aug 2026 Tencent Hy4 preview 66.6 ArXivMath
2 Jul 2026 Tencent Hy3 52.2 ArxivMath
9 Jun 2026 Claude Fable 5 78.5 ArxivMath

5 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
28 Sep 2026 Claude Sonnet 5.5 system card 86.8 ArXivMath August 2026 release, max effort without tools (95.2% with tools), four attempts per problem (8.9, Figure 8.9.A) Capabilities
22 Sep 2026 Claude Opus 5.5 system card 91.2 ArXivMath August 2026 release (57 problems), max effort without tools (96.9% with tools), four attempts per problem (8.9, Figures 8.9.A and B) Capabilities
24 Jul 2026 Claude Opus 5 system card 90.8 ArXivMath June 2026 release (49 problems), max effort without tools (91.3% with tools), four runs per problem (8.8) Capabilities
30 Jun 2026 Claude Sonnet 5 system card 65.7 ArXivMath April and May 2026 releases (81 problems), extended thinking, without tools (72.2% with tools), four attempts per problem (8.7, Figure 8.7.A) Capabilities
28 May 2026 Claude Opus 4.8 system card 71.8 ArXivMath March and April 2026 releases, 71 problems, extended thinking (8.8) Capabilities

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%