Benchmark Almanac

BrokenArXiv

GPT-6.1 Sol 89.7% leads since Sep 2026 In use

Models are asked to prove statements from recent arXiv papers that were changed to be plausible but false. Full credit goes to models that say the statement is false.

MathArena releases a new set each month. The August 2026 set brought harder disproven conjectures, tool harnesses and a 0 to 3 grade. Scores from before and after that change do not compare.

Made by MathArena (ETH Zurich SRI Lab and INSAIT) Released Mar 2026 Research mathematics All tasks public Last in a launch post Aug 2026 Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Research frontier Researchers, on open problems or tasks that take days
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Written proof
eth-sri/matharena 294 stars Dataset Leaderboard matharena.ai
6
Models scored on its own leaderboard
1
Launch post lists it, from 1 lab

GPT-6.1 Sol leads with 89.7%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.

  1. 89.7 GPT-6.1 Sol
  2. 84.7 Claude Opus 5.5
  3. 49.6 DeepSeek V4.1 Flash
  4. 38.5 Grok 4.7
  5. 33.0 Muse Spark Open
  6. 25.0 Gemini 3.8 Flash
  1. 1 GPT-6.1 Sol max 89.7%
  2. 2 Claude Opus 5.5 high 84.7%
  3. 3 DeepSeek V4.1 Flash max 49.6%
  4. 4 Grok 4.7 xhigh 38.5%
  5. 5 Muse Spark Open 33%
  6. 6 Gemini 3.8 Flash 25%
#ModelScoreSettingDateSource
1 GPT-6.1 Sol 89.7% max Sep 2026 Official leaderboard
2 Claude Opus 5.5 84.7% high Sep 2026 Official leaderboard
3 DeepSeek V4.1 Flash 49.6% max Sep 2026 Official leaderboard
4 Grok 4.7 38.5% xhigh Sep 2026 Official leaderboard
5 Muse Spark Open 33% Oct 2026 Official leaderboard
6 Gemini 3.8 Flash 25% Sep 2026 Official leaderboard

Labs report scores for 1 model in their cards. We plot every source in the charts below.

4 models have held number one since its release in Mar 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind DeepSeek Anthropic OpenAI
Own leaderboard Passed 85% in Sep 2026, by GPT-6.1 Sol
GPT-6.1 Sol 89.7%

The best score passed 85% in Sep 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% Near the ceiling (85%) GPT-6.1 Sol Claude Opus 5.5 Tencent Hy4 preview 0% 20% 40% 60% 80% 100% Near the ceiling (85%) GPT-6.1 Sol Tencent Hy4 preview

One lab put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Tencent
DateLaunch postScoreAs the post writes it
28 Aug 2026 Tencent Hy4 preview 54.6 BrokenArXiv

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%