Benchmark Almanac

FrontierMath Erdős

GPT-6.1 Sol 4.41% leads since Sep 2026 Not in model cards

Models try to prove or disprove 68 open Erdős problems in Lean 4. A result counts only if the Lean checker accepts the full proof.

By late October 2026, 17 of the 68 conjectures had been resolved by Epoch's runs or by others.

In the authors’ words

We introduce FrontierMath Erdős (FME), a benchmark of 68 Erdős problems that are open as of August 2026. To solve a task in FME, AI systems must resolve (prove or disprove) one of the 68 conjectures in the proof assistant Lean. Our 68 problems were selected by the second author among 652 open problems on erdosproblems.com for their mathematical interest and difficulty. AIs have recently resolved several open problems in mathematics, but these demonstrations fall short of a systematic study of AI capabilities. FME evaluates every AI model on the same fixed problems, autonomously and under the same budget. We evaluated five AIs with a budget of $300 per problem. One (GPT-6 Astra) scored 3%, and all others scored 0%.

Abstract of FrontierMath Erdős (arXiv:2609.25050, CC0)
Made by Epoch AI Released Sep 2026 Research mathematics All tasks public Last model added Sep 2026 Scores refreshed 31 Oct 2026
Level
Research frontier Researchers, on open problems or tasks that take days
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Written proof
epoch-research/LeanOpenProblems 18 stars Paper Leaderboard epoch.ai
14
Models scored in Epoch AI data

GPT-6.1 Sol leads with 4.41%

Best score per model in Epoch AI’s data. Where a model ran with several settings or harnesses, we keep its best.

14 models. Scroll sideways for the rest.

  1. 4.41 GPT-6.1 Sol
  2. 2.94 GPT-6 Astra
  3. 2.94 Claude Opus 5.5
  4. 2.90 Claude Sonnet 5.5
  5. 1.47 GPT-6 Sol
  6. 1.47 Gemini 4 Argon
  7. 0.0 GPT-5.5
  8. 0.0 Claude Fable 5
  9. 0.0 GPT-5.6 Sol
  10. 0.0 Kimi K3
  11. 0.0 Claude Opus 5
  12. 0.0 Claude Fable 5.1
  13. 0.0 Gemini 3.8 Flash
  14. 0.0 Grok 4.7
  1. 1 GPT-6.1 Sol max 4.41%
  2. 2 GPT-6 Astra max 2.94%
  3. 3 Claude Opus 5.5 max 2.94%
  4. 4 Claude Sonnet 5.5 max 2.90%
  5. 5 GPT-6 Sol max 1.47%
  6. 6 Gemini 4 Argon high 1.47%
  7. 7 GPT-5.5 xhigh 0%
  8. 8 Claude Fable 5 max 0%
  9. 9 GPT-5.6 Sol max 0%
  10. 10 Kimi K3 max 0%
  11. 11 Claude Opus 5 max 0%
  12. 12 Claude Fable 5.1 max 0%
  13. 13 Gemini 3.8 Flash high 0%
  14. 14 Grok 4.7 xhigh 0%
#ModelScoreSettingDateSource
1 GPT-6.1 Sol 4.41% max Sep 2026 Epoch AI
2 GPT-6 Astra 2.94% max Sep 2026 Epoch AI
3 Claude Opus 5.5 2.94% max Sep 2026 Epoch AI
4 Claude Sonnet 5.5 2.90% max Sep 2026 Epoch AI
5 GPT-6 Sol 1.47% max Sep 2026 Epoch AI
6 Gemini 4 Argon 1.47% high Sep 2026 Epoch AI
7 GPT-5.5 0% xhigh Apr 2026 Epoch AI
8 Claude Fable 5 0% max Jun 2026 Epoch AI
9 GPT-5.6 Sol 0% max Jul 2026 Epoch AI
10 Kimi K3 0% max Jul 2026 Epoch AI
11 Claude Opus 5 0% max Jul 2026 Epoch AI
12 Claude Fable 5.1 0% max Sep 2026 Epoch AI
13 Gemini 3.8 Flash 0% high Sep 2026 Epoch AI
14 Grok 4.7 0% xhigh Sep 2026 Epoch AI

3 models have held number one since its release in Sep 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Epoch AI runs 3 leaders
GPT-6.1 Sol 4.41%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Run or collected by Epoch AI Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-6.1 Sol GPT-5.5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-6.1 Sol

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%