PutnamBench
Leanstral 1.5 87.5% leads the models we track Fading Models write formal Lean proofs of Putnam competition problems. A proof counts only when the verifier accepts it.
The Lean set now has 672 problems, and five systems solved all of them by September 2026, so the board ranks full solvers by cost. Labs often report Putnam-200, a fixed subset first used by Seed-Prover.
In the authors’ words
We present PutnamBench, a new multi-language benchmark for evaluating the ability of neural theorem-provers to solve competition mathematics problems. PutnamBench consists of 1692 hand-constructed formalizations of 640 theorems sourced from the William Lowell Putnam Mathematical Competition, the premier undergraduate-level mathematics competition in North America. All the problems have formalizations in Lean 4 and Isabelle; a substantial subset also has Coq formalizations. PutnamBench requires significant problem-solving ability and proficiency in a broad range of topics taught in undergraduate mathematics courses. We use PutnamBench to evaluate several established neural and symbolic theorem-provers. These approaches can only solve a handful of the PutnamBench problems, establishing the benchmark as a difficult open challenge for research on neural theorem-proving. PutnamBench is available at https://github.com/trishullab/PutnamBench.
- Level
- Competition The strongest students, in olympiad-style contests
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Written proof
Aleph Prover (Logical Intelligence) tops the leaderboard with 100%
Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Leanstral 1.5, at 87.5%.
The newest model with a score here came out in Jul 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 16 more
- 100.0 Aleph Prover (Logical Intelligence)
- 100.0 Humanfia
- 100.0 NEAR AI (w/ DeepSeek V4)
- 100.0 Forall (Astrio)
- 97.9 Mistral Prover (DeepSeek V4 Pro + Leanstral)
- 95.5 Goedel-Architect
- 87.5 Leanstral 1.5
- 86.5
Seed-Prover 1.5 - 68.8 Hilbert
- 54.3 AxProverBase (Axiomatic AI)
- 49.0 Seed-Prover
- 16.4 C2C-Goedel
- 13.5 Ax-Prover (Axiomatic AI)
- 12.8 Goedel-Prover-V2
- 9.67 Ensemble Prover (Graviterra)
- 6.99
DeepSeek-Prover-V2-671B - 4.17 GPT-5 (ReAct, 10 turns)
- 3.72 C2C-Kimina
- 3.42 DSP+
- 2.08 Bourbaki
- 1.49 Kimina-Prover-7B-Distill
- 1.19 Self-play Theorem Prover
- 1.04 ABEL
- 1.04 Goedel-Prover-SFT
- 0.89 InternLM2.5-StepProver
- 0.60 InternLM-Math 7B
- 0.45
Gemini 2.5 Pro - 0.30 o4-mini
- 0.15 GPT-4o
- 0.15 COPRA (GPT-4o)
- 0.15 DeepSeek-R1
- 0.15 Gemini 2.0 Flash Thinking
- 0.0 ReProver w/ retrieval
- 0.0 ReProver w/o retrieval
- 0.0 GPT-4o mini
- 0.0 o3-mini
- 0.0 Claude 3.7 Sonnet
- 0.0 DeepSeek-V3
- 0.0 Grok-3 mini
- 1 Aleph Prover (Logical Intelligence) 100%
- 2 Humanfia 100%
- 3 NEAR AI (w/ DeepSeek V4) 100%
- 4 Forall (Astrio) 100%
- 5 Mistral Prover (DeepSeek V4 Pro + Leanstral) 97.9%
- 6 Goedel-Architect 95.5%
- 7 Leanstral 1.5 87.5%
- 8 Seed-Prover 1.5 86.5%
- 9 Hilbert 68.8%
- 10 AxProverBase (Axiomatic AI) 54.3%
- 11 Seed-Prover 49%
- 12 C2C-Goedel 16.4%
- 13 Ax-Prover (Axiomatic AI) 13.5%
- 14 Goedel-Prover-V2 12.8%
- 15 Ensemble Prover (Graviterra) 9.67%
- 16 DeepSeek-Prover-V2-671B 6.99%
- 17 GPT-5 (ReAct, 10 turns) 4.17%
- 18 C2C-Kimina 3.72%
- 19 DSP+ 3.42%
- 20 Bourbaki 2.08%
- 21 Kimina-Prover-7B-Distill 1.49%
- 22 Self-play Theorem Prover 1.19%
- 23 ABEL 1.04%
- 24 Goedel-Prover-SFT 1.04%
- 25 InternLM2.5-StepProver 0.89%
- 26 InternLM-Math 7B 0.60%
- 27 Gemini 2.5 Pro Mar 2025 0.45%
- 28 o4-mini high 0.30%
- 29 GPT-4o May 2024 0.15%
- 30 COPRA (GPT-4o) 0.15%
- 31 DeepSeek-R1 0.15%
- 32 Gemini 2.0 Flash Thinking Jan 2025 0.15%
- 33 ReProver w/ retrieval 0%
- 34 ReProver w/o retrieval 0%
- 35 GPT-4o mini 0%
- 36 o3-mini 0%
- 37 Claude 3.7 Sonnet 0%
- 38 DeepSeek-V3 Mar 2025 0%
- 39 Grok-3 mini 0%
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Aleph Prover (Logical Intelligence) | 100% | Aug 2026 | Official leaderboard | |
| 2 | Humanfia | 100% | Aug 2026 | Official leaderboard | |
| 3 | NEAR AI (w/ DeepSeek V4) | 100% | Sep 2026 | Official leaderboard | |
| 4 | Forall (Astrio) | 100% | Sep 2026 | Official leaderboard | |
| 5 | Mistral Prover (DeepSeek V4 Pro + Leanstral) | 97.9% | Sep 2026 | Official leaderboard | |
| 6 | Goedel-Architect | 95.5% | Jul 2026 | Official leaderboard | |
| 7 | Leanstral 1.5 | 87.5% | Jul 2026 | Official leaderboard | |
| 8 | Seed-Prover 1.5 | 86.5% | Dec 2025 | Official leaderboard | |
| 9 | Hilbert | 68.8% | Oct 2025 | Official leaderboard | |
| 10 | AxProverBase (Axiomatic AI) | 54.3% | Apr 2026 | Official leaderboard | |
| 11 | Seed-Prover | 49% | Jul 2025 | Official leaderboard | |
| 12 | C2C-Goedel | 16.4% | Aug 2026 | Official leaderboard | |
| 13 | Ax-Prover (Axiomatic AI) | 13.5% | Dec 2025 | Official leaderboard | |
| 14 | Goedel-Prover-V2 | 12.8% | Jul 2025 | Official leaderboard | |
| 15 | Ensemble Prover (Graviterra) | 9.67% | Sep 2026 | Official leaderboard | |
| 16 | DeepSeek-Prover-V2-671B | 6.99% | Apr 2025 | Official leaderboard | |
| 17 | GPT-5 (ReAct, 10 turns) | 4.17% | Nov 2025 | Official leaderboard | |
| 18 | C2C-Kimina | 3.72% | Aug 2026 | Official leaderboard | |
| 19 | DSP+ | 3.42% | Jun 2025 | Official leaderboard | |
| 20 | Bourbaki | 2.08% | Jul 2025 | Official leaderboard | |
| 21 | Kimina-Prover-7B-Distill | 1.49% | Apr 2025 | Official leaderboard | |
| 22 | Self-play Theorem Prover | 1.19% | Feb 2025 | Official leaderboard | |
| 23 | ABEL | 1.04% | Oct 2024 | Official leaderboard | |
| 24 | Goedel-Prover-SFT | 1.04% | Jan 2025 | Official leaderboard | |
| 25 | InternLM2.5-StepProver | 0.89% | Nov 2024 | Official leaderboard | |
| 26 | InternLM-Math 7B | 0.60% | Jul 2024 | Official leaderboard | |
| 27 | Gemini 2.5 Pro Mar 2025 | 0.45% | Mar 2025 | Official leaderboard | |
| 28 | o4-mini | 0.30% | high | Apr 2025 | Official leaderboard |
| 29 | GPT-4o May 2024 | 0.15% | May 2024 | Official leaderboard | |
| 30 | COPRA (GPT-4o) | 0.15% | Jul 2024 | Official leaderboard | |
| 31 | DeepSeek-R1 | 0.15% | Jan 2025 | Official leaderboard | |
| 32 | Gemini 2.0 Flash Thinking Jan 2025 | 0.15% | Jan 2025 | Official leaderboard | |
| 33 | ReProver w/ retrieval | 0% | Jul 2024 | Official leaderboard | |
| 34 | ReProver w/o retrieval | 0% | Jul 2024 | Official leaderboard | |
| 35 | GPT-4o mini | 0% | Jul 2024 | Official leaderboard | |
| 36 | o3-mini | 0% | Jan 2025 | Official leaderboard | |
| 37 | Claude 3.7 Sonnet | 0% | Feb 2025 | Official leaderboard | |
| 38 | DeepSeek-V3 Mar 2025 | 0% | Mar 2025 | Official leaderboard | |
| 39 | Grok-3 mini | 0% | Apr 2025 | Official leaderboard |
Labs report scores for 7 models in their cards. We plot every source in the charts below.
6 models have held number one since its release in Jul 2024
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
The best score passed 85% in Dec 2025
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Labs headline it less often now, last in Jul 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 2 Jul 2026 | Leanstral 1.5 | 87.4 | PutnamBench 587/672 solved (pass@8, 4M tokens per attempt) |
| 14 Feb 2026 | Seed2.0 Pro | 35.5 | Putnam-200 Pass@8, agentic Lean setup (Table 5) |
| 19 Dec 2025 | Seed-Prover 1.5 | 87.9 | PutnamBench 580/660 (87.9%) |
| 31 Jul 2025 | Seed-Prover | 50.4 | PutnamBench 331/657 (Medium) |
| 30 Apr 2025 | DeepSeek-Prover-V2-671B | 7.4 | PutnamBench 49 out of 658 problems solved |
5 system cards report it, from 4 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 31 Jul 2026 | DiffusionGemma 26B A4B technical report | 67.4 | Putnam, cited as PutnamBench (Tsoukalas et al., 2024), TD, 57.1 no-think, scoring setup not stated (Table 3) | Capabilities |
| 2 Jul 2026 | Leanstral 1.5 technical report | 87.4 | PutnamBench 587/672 solved, pass@8, 4M-token limit per trajectory, $1.68 per problem (Table 1) | Capabilities |
| 26 Apr 2026 | DeepSeek-V4-Flash technical report | 81 | Putnam-200 Pass@8, fixed random subset of PutnamBench, agentic Lean setting with LeanExplore, DeepSeek-V4-Flash-Max (Figure 8 left) | Capabilities |
| 14 Feb 2026 | Seed2.0 Pro model card | 35.5 | Putnam-200, a 200-problem PutnamBench subset, Pass@8, agentic Lean setup (Table 5) | Capabilities |
| 30 Apr 2025 | DeepSeek-Prover-V2-671B technical report | 7.14 | PutnamBench, CoT, sample budget 1024, 47 of 658 solved (Table 4). Version 1 reported 49 | Capabilities |
More in Math
- IMO-AnswerBench In use 400 olympiad problems with short verifiable answers, robustified against memorization. Scored by answer match. GLM-5.2 91%
- AIME 2026 In use Models solve the 30 integer-answer problems from the 2026 AIME I and II math competitions. Scored by exact answer match. GPT-5.5 100%
- HMMT 2026 In use Models solve the 33 final-answer problems from the February 2026 Harvard-MIT Mathematics Tournament. Scored by exact answer match. GPT-5.5 98.5%
- MathArena (Apex, IMO, USAMO) In use Evaluates models on newly released math competitions after release. Final-answer or human/LLM-judged proof scores. Claude Opus 4.8 81.3%
- PolyMath In use Models solve 9,000 parallel math problems in 18 languages across four difficulty levels, from school math to frontier math. Final answers are checked.
Qwen3.7-Max 86.5% - USAMO 2026 In use Models write full proofs for the six problems of the 2026 USA Mathematical Olympiad. Judges grade each proof from 0 to 7 against rubrics. GPT-5.5 98.2%