Benchmark Almanac

ProofBench v1.1

Claude Fable 5.1 100% leads since Sep 2026 Not in model cards

Models write Lean 4 proofs of graduate-level theorems from paired natural-language and formal statements. Scored by the share the Lean checker accepts.

Vals says scores are not comparable with earlier revisions. Four systems tied at 100% on 2026-10-29 among 54, so it is saturated.

Made by Vals AI Released Aug 2026 Formal proofs Public tasks plus a private test set Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Written proof
Paper Leaderboard
53
Entries on its own leaderboard, 51 models we track

3 models share the top score of 100%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

53 entries. Scroll sideways for the rest.

  1. 100.0 AlephProver
  2. 100.0 Claude Fable 5.1
  3. 100.0 Claude Opus 5.5
  4. 100.0 Claude Sonnet 5.5
  5. 99.0 Claude Opus 5
  6. 99.0 GPT-6 Astra
  7. 99.0 GPT-6.1 Sol
  8. 99.0 Gemini 4 Argon
  9. 95.0 Claude Fable 5
  10. 94.0 Muse Wonder Full
  11. 91.0 Kimi K3.1 Pro
  12. 89.0 Claude Haiku 5.5
  13. 87.0 Kimi K3
  14. 86.0 Aristotle
  15. 84.0 Muse Wonder
  16. 83.0 GPT-5.6 Sol
  17. 83.0 GPT-6 Sol
  18. 77.0 Claude Sonnet 5
  19. 75.0 Tencent Hy4 preview
  20. 74.0 GPT-5.6 Terra
  21. 70.0 MiMo-V2.6 Pro
  22. 64.0 GPT-6 Luna
  23. 63.0 MiMo-V2.6 Flash
  24. 61.0 Ember-1
  25. 60.0 GPT-5.6 Luna
  26. 58.0 Qwen 3.8 Max
  27. 58.0 Gemini 3.7 Flash
  28. 58.0 Muse Spark 1.3
  29. 56.0 DeepSeek V4 Flash 0731
  30. 54.0 DeepSeek V4.1 Flash
  31. 51.0 Grok 4.6
  32. 50.0 DeepSeek V4 Pro 0813
  33. 49.0 GLM-5.3
  34. 48.0 Gemini 3.8 Flash
  35. 43.0 Muse Spark 1.2
  36. 42.0 Step 5 Preview
  37. 31.0 Gemini 3.5 Flash
  38. 31.0 Grok 4.5
  39. 26.0 Gemini 3.1 Pro
  40. 26.0 Grok 4.7
  41. 22.0 MiMo-V2.5-Pro
  42. 21.0 GLM-5.3-Flash
  43. 18.0 MiniMax-M3
  44. 18.0 Beam 1.0
  45. 16.0 MiMo-V2.5
  46. 16.0 DeepSeek-V4-Pro
  47. 16.0 Qwen 3.8 27B
  48. 10.0 Mistral Large 4
  49. 9.00 Mistral Medium 3.5
  50. 6.00 Inkling-Small
  51. 3.00 Mercury 2.5
  52. 0.0 Inkling
  53. 0.0 Nemotron 3.5 Lightning 30B-A3B
  1. 1 AlephProver 100%
  2. 2 Claude Fable 5.1 100%
  3. 3 Claude Opus 5.5 100%
  4. 4 Claude Sonnet 5.5 100%
  5. 5 Claude Opus 5 99%
  6. 6 GPT-6 Astra 99%
  7. 7 GPT-6.1 Sol 99%
  8. 8 Gemini 4 Argon 99%
  9. 9 Claude Fable 5 95%
  10. 10 Muse Wonder Full 94%
  11. 11 Kimi K3.1 Pro 91%
  12. 12 Claude Haiku 5.5 89%
  13. 13 Kimi K3 87%
  14. 14 Aristotle 86%
  15. 15 Muse Wonder 84%
  16. 16 GPT-5.6 Sol 83%
  17. 17 GPT-6 Sol 83%
  18. 18 Claude Sonnet 5 77%
  19. 19 Tencent Hy4 preview 75%
  20. 20 GPT-5.6 Terra 74%
  21. 21 MiMo-V2.6 Pro 70%
  22. 22 GPT-6 Luna 64%
  23. 23 MiMo-V2.6 Flash 63%
  24. 24 Ember-1 61%
  25. 25 GPT-5.6 Luna 60%
  26. 26 Qwen 3.8 Max 58%
  27. 27 Gemini 3.7 Flash 58%
  28. 28 Muse Spark 1.3 max 58%
  29. 29 DeepSeek V4 Flash 0731 56%
  30. 30 DeepSeek V4.1 Flash 54%
  31. 31 Grok 4.6 51%
  32. 32 DeepSeek V4 Pro 0813 50%
  33. 33 GLM-5.3 49%
  34. 34 Gemini 3.8 Flash 48%
  35. 35 Muse Spark 1.2 43%
  36. 36 Step 5 Preview 42%
  37. 37 Gemini 3.5 Flash 31%
  38. 38 Grok 4.5 31%
  39. 39 Gemini 3.1 Pro 26%
  40. 40 Grok 4.7 26%
  41. 41 MiMo-V2.5-Pro 22%
  42. 42 GLM-5.3-Flash 21%
  43. 43 MiniMax-M3 18%
  44. 44 Beam 1.0 18%
  45. 45 MiMo-V2.5 16%
  46. 46 DeepSeek-V4-Pro 16%
  47. 47 Qwen 3.8 27B 16%
  48. 48 Mistral Large 4 10%
  49. 49 Mistral Medium 3.5 9.00%
  50. 50 Inkling-Small 6.00%
  51. 51 Mercury 2.5 3.00%
  52. 52 Inkling 0%
  53. 53 Nemotron 3.5 Lightning 30B-A3B 0%
#EntryScoreSettingDateSource
1 AlephProver 100% Official leaderboard
2 Claude Fable 5.1 100% Sep 2026 Official leaderboard
3 Claude Opus 5.5 100% Sep 2026 Official leaderboard
4 Claude Sonnet 5.5 100% Sep 2026 Official leaderboard
5 Claude Opus 5 99% Jul 2026 Official leaderboard
6 GPT-6 Astra 99% Sep 2026 Official leaderboard
7 GPT-6.1 Sol 99% Sep 2026 Official leaderboard
8 Gemini 4 Argon 99% Sep 2026 Official leaderboard
9 Claude Fable 5 95% Jun 2026 Official leaderboard
10 Muse Wonder Full 94% Oct 2026 Official leaderboard
11 Kimi K3.1 Pro 91% Oct 2026 Official leaderboard
12 Claude Haiku 5.5 89% Oct 2026 Official leaderboard
13 Kimi K3 87% Jul 2026 Official leaderboard
14 Aristotle 86% Official leaderboard
15 Muse Wonder 84% Oct 2026 Official leaderboard
16 GPT-5.6 Sol 83% Jul 2026 Official leaderboard
17 GPT-6 Sol 83% Sep 2026 Official leaderboard
18 Claude Sonnet 5 77% Jun 2026 Official leaderboard
19 Tencent Hy4 preview 75% Aug 2026 Official leaderboard
20 GPT-5.6 Terra 74% Jul 2026 Official leaderboard
21 MiMo-V2.6 Pro 70% Sep 2026 Official leaderboard
22 GPT-6 Luna 64% Sep 2026 Official leaderboard
23 MiMo-V2.6 Flash 63% Sep 2026 Official leaderboard
24 Ember-1 61% Sep 2026 Official leaderboard
25 GPT-5.6 Luna 60% Jul 2026 Official leaderboard
26 Qwen 3.8 Max 58% Aug 2026 Official leaderboard
27 Gemini 3.7 Flash 58% Aug 2026 Official leaderboard
28 Muse Spark 1.3 58% max Sep 2026 Official leaderboard
29 DeepSeek V4 Flash 0731 56% Jul 2026 Official leaderboard
30 DeepSeek V4.1 Flash 54% Sep 2026 Official leaderboard
31 Grok 4.6 51% Aug 2026 Official leaderboard
32 DeepSeek V4 Pro 0813 50% Aug 2026 Official leaderboard
33 GLM-5.3 49% Aug 2026 Official leaderboard
34 Gemini 3.8 Flash 48% Sep 2026 Official leaderboard
35 Muse Spark 1.2 43% Aug 2026 Official leaderboard
36 Step 5 Preview 42% Sep 2026 Official leaderboard
37 Gemini 3.5 Flash 31% May 2026 Official leaderboard
38 Grok 4.5 31% Jul 2026 Official leaderboard
39 Gemini 3.1 Pro 26% Feb 2026 Official leaderboard
40 Grok 4.7 26% Sep 2026 Official leaderboard
41 MiMo-V2.5-Pro 22% Apr 2026 Official leaderboard
42 GLM-5.3-Flash 21% Aug 2026 Official leaderboard
43 MiniMax-M3 18% Jun 2026 Official leaderboard
44 Beam 1.0 18% Oct 2026 Official leaderboard
45 MiMo-V2.5 16% Apr 2026 Official leaderboard
46 DeepSeek-V4-Pro 16% Apr 2026 Official leaderboard
47 Qwen 3.8 27B 16% Aug 2026 Official leaderboard
48 Mistral Large 4 10% Oct 2026 Official leaderboard
49 Mistral Medium 3.5 9.00% Apr 2026 Official leaderboard
50 Inkling-Small 6.00% Jul 2026 Official leaderboard
51 Mercury 2.5 3.00% Sep 2026 Official leaderboard
52 Inkling 0% Jul 2026 Official leaderboard
53 Nemotron 3.5 Lightning 30B-A3B 0% Aug 2026 Official leaderboard

The best score, 100%, costs $0.56 a task

Runs on the cost frontier score higher than any run that's cheaper. GPT-6.1 Sol comes within 1 points of the best for $0.23 a task, about half the cost. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

NVIDIA OpenAI Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Sonnet 5.5 GPT-6.1 Sol Claude Haiku 5.5 GPT-6 Luna Nemotron 3.5 Lightning 30B-A3B 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Sonnet 5.5 Claude Haiku 5.5
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.1 100% $2.70
Claude Opus 5.5 100% $0.96
Claude Sonnet 5.5 100% $0.56
Claude Opus 5 99% $1.79
GPT-6 Astra 99% $1.71
GPT-6.1 Sol 99% $0.23
Gemini 4 Argon 99% $0.78
Claude Fable 5 95% $5.23
Muse Wonder Full 94% $0.86
Kimi K3.1 Pro 91% $0.47
Claude Haiku 5.5 89% $0.17
Kimi K3 87% $1.65
Muse Wonder 84% $1.31
GPT-5.6 Sol 83% $1.44
GPT-6 Sol 83% $0.48
Claude Sonnet 5 77% $1.37
Tencent Hy4 preview 75% $0.34
GPT-5.6 Terra 74% $1.27
MiMo-V2.6 Pro 70% $0.25
GPT-6 Luna 64% $0.04
MiMo-V2.6 Flash 63% $0.12
Ember-1 61% $1.74
GPT-5.6 Luna 60% $0.12
Qwen 3.8 Max 58% $1.50
Gemini 3.7 Flash 58% $0.56
Muse Spark 1.3 max 58% $0.48
DeepSeek V4 Flash 0731 56% $0.16
Muse Spark 1.3 55% $0.43
DeepSeek V4.1 Flash 54% $0.13
Grok 4.6 51% $0.82
DeepSeek V4 Pro 0813 50% $0.26
GLM-5.3 49% $2.08
Gemini 3.8 Flash 48% $0.60
Muse Spark 1.2 43% $0.44
Step 5 Preview 42% $0.71
Gemini 3.5 Flash 31% $0.57
Grok 4.5 31% $0.57
Gemini 3.1 Pro 26% $0.78
Grok 4.7 26% $0.79
MiMo-V2.5-Pro 22% $0.09
GLM-5.3-Flash 21% $0.22
MiniMax-M3 18% $0.42
Beam 1.0 18%
MiMo-V2.5 16% $0.06
DeepSeek-V4-Pro 16% $0.14
Qwen 3.8 27B 16% $2.30
Mistral Large 4 10% $0.33
Mistral Medium 3.5 9.00% $1.30
Inkling-Small 6.00% $0.08
Mercury 2.5 3.00% $0.04
Inkling 0% $0.20
Nemotron 3.5 Lightning 30B-A3B 0% $0.02

2 models have held number one since its release in Aug 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Passed 85% in Aug 2026, by Claude Opus 5
Claude Fable 5.1 100%

The best score passed 85% in Jun 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Fable 5.1 Claude Fable 5 Gemini 3.5 Flash Gemini 3.1 Pro 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Fable 5.1 Gemini 3.5 Flash

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic OpenAI
ProofBench v1.1 Passed 85% in Aug 2026, by Claude Opus 5
Claude Fable 5.1 100%
ProofBench Passed 85% in Jun 2026, by Claude Fable 5
Claude Fable 5.1 100%
ProofBench v1.1 Passed 85% in Aug 2026, by Claude Opus 5
Claude Fable 5.1 100%
Replaces ProofBench

More in Math

All 105 in Math
  • IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
  • AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
  • HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
  • MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
  • PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked. Qwen3.7-Max 86.5%
  • USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%