Benchmark Almanac

o4-mini

o4-mini ranks first on 4 of the 154 benchmarks it has scores on.

OpenAI Released 16 Apr 2025
158
Benchmarks with a score
4
Ranked first, out of five or more models
17
In the top three
OpenAI Apr 2025

o4-mini

Closed weights

145.6 #90 of 271
Epoch Capabilities Index
  • Humanity's Last Exam 18.1% #23 /47
  • Toolathlon 14.8% #44 /50
  • MultiChallenge 44.9% #32 /35

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
Arena-Hard v2.0 79.1% #2 of 23 Official leaderboard
Fiction.LiveBench 77.8% #12 of 57 Epoch AI
Vectara Hallucination Leaderboard (HHEM) 6.80% #40 of 148 Official leaderboard
Wer wird Millionär? LLM Benchmark 56178.0 #25 of 76 Official leaderboard
ECLeKTic 22.6% #12 of 35 Official leaderboard
LMArena (Elo) 1,391 #139 of 333 Official leaderboard
LLM Confabulation Leaderboard 15.8% #26 of 58 Official leaderboard
WritingBench 72.9 #21 of 45 Official leaderboard
EQ-Bench 3 1,279 #37 of 78 Official leaderboard
Humanity's Last Exam 18.1% #23 of 47 Official leaderboard
MMMLU 85.2% #32 of 62 Model card
Global MMLU Lite 87% #27 of 43 Official leaderboard
FACTS Search 66.2% #20 of 31 Official leaderboard
Lech Mazur Creative Writing 7.50 #29 of 44 Epoch AI
SimpleQA 20% #39 of 59 Model card
PersonQA 36% #6 of 9 Model card
FACTS Multimodal 34.5% #27 of 39 Official leaderboard
SimpleQA Verified 21.2% #39 of 56 Official leaderboard
FaithJudge 25.8% #26 of 37 Official leaderboard
MultiNRC 22.2% #27 of 38 Official leaderboard
FACTS Parametric 20.5% #33 of 45 Official leaderboard
IndicGenBench 41.9 #10 of 13 Official leaderboard
FACTS Benchmark Suite 34.5% #33 of 40 Official leaderboard
Judgemark v2.1 56.1 #35 of 42 Official leaderboard
BullshitBench 10.9% #96 of 107 Official leaderboard
BullshitBench V2 8.00% #110 of 122 Official leaderboard
FACTS Grounding 62.1% #31 of 34 Official leaderboard
MultiChallenge 44.9% #32 of 35 Official leaderboard
SimpleQA Hallucination Rate 79% #11 of 12 Model card
Vectara Hallucination Leaderboard (2025 dataset) 18.6% #97 of 104 Official leaderboard

Math and science

BenchmarkScoreRankSource
MATH Level 5 97.8% #3 of 94 Epoch AI
AIME 2024 93.4% #4 of 87 Model card
PolyMath 45.6% #9 of 38 Official leaderboard
AMO-Bench 40.2% #5 of 21 Official leaderboard
AIME 2025 92.7% #37 of 143 Model card
USAMO 2025 19.1% #3 of 10 Official leaderboard
SMT 2025 88.7% #13 of 40 Official leaderboard
FrontierMath 24.8% #25 of 70 Epoch AI
HiPhO 59.5% #13 of 35 Official leaderboard
PHYBench 29.4% #11 of 28 Official leaderboard
HMMT 2025 83.3% #21 of 53 Official leaderboard
GPQA Diamond 79.6% #89 of 219 Epoch AI
CORE-Bench 35.6% #7 of 17 Official leaderboard
FrontierMath Tier 4 6.25% #23 of 55 Epoch AI
OTIS Mock AIME 2024-2025 81.7% #86 of 202 Epoch AI
PutnamBench 0.30% #6 of 14 Official leaderboard
DTBench 62.7% #96 of 178 Official leaderboard
CMIMC 2025 83.8% #18 of 33 Official leaderboard
Misguided Attention 42.9% #25 of 45 Official leaderboard
CritPt 0.60% #101 of 160 Epoch AI
ARC-AGI-2 6.11% #61 of 95 Official leaderboard
ForecastBench 63.7 #56 of 86 Official leaderboard
ARC-AGI 58.7% #60 of 92 Official leaderboard
General365 51% #15 of 23 Official leaderboard
BRUMO 2025 86.7% #28 of 42 Official leaderboard
IMO 2025 14.3% #4 of 6 Official leaderboard
FrontierMath Tiers 1-3 v2 36.1% #63 of 89 Epoch AI
SimpleBench 38.7% #71 of 100 Official leaderboard
IMProofBench 6.45% #8 of 10 Official leaderboard
FrontierMath Tier 4 v2 4.88% #61 of 71 Epoch AI
miniCTX-v2 24.9% #4 of 4 Official leaderboard

Coding

BenchmarkScoreRankSource
LiveCodeBench v5 81.4% #1 of 21 Official leaderboard
LiveCodeBench v6 80.2% #1 of 21 Official leaderboard
OpenAI Research Engineer Interview: Multiple Choice 83% #1 of 9 Model card
Codeforces rating 2,719 #7 of 41 Model card
AutoCodeBench 50% #6 of 34 Official leaderboard
Aider Polyglot 72% #10 of 54 Epoch AI
OpenAI Research Engineer Interview: Coding 98% #2 of 7 Model card
AlgoTune 1.72× #6 of 18 Official leaderboard
SWE-bench Multimodal 33.9% #3 of 9 Official leaderboard
WeirdML 52.6% #44 of 112 Official leaderboard
SWE-bench Verified 64.6% #24 of 60 Official leaderboard
METR Time Horizon 77 min #16 of 39 Epoch AI
LiveSQLBench 24.2% #19 of 36 Official leaderboard
ALE-Bench 826 #66 of 121 Epoch AI
SWE-Lancer 52% #6 of 10 Model card
PaperBench 25% #9 of 14 Model card
OpenAI PRs 39% #8 of 12 Model card
IOI (Vals) 4.83% #42 of 55 Official leaderboard
GSO-Bench 3.60% #27 of 30 Official leaderboard
SWE-Lancer Diamond (SWE Manager) 23% #4 of 4 Model card

Agents and tools

BenchmarkScoreRankSource
SnakeBench 35 #11 of 183 Official leaderboard
BFCL V4 53.2% #17 of 65 Official leaderboard
LLM Chess 440 #44 of 124 Official leaderboard
Galileo Agent Leaderboard 0.89 #12 of 31 Official leaderboard
Step Race 3 #26 of 59 Official leaderboard
LisanBench 800.0 #61 of 133 Official leaderboard
MCP-Universe 26% #14 of 30 Official leaderboard
VITA-Bench 19.5% #10 of 18 Official leaderboard
GAIA 42.5% #34 of 57 Official leaderboard
Chess Puzzles 22% #55 of 83 Epoch AI
AgentBench FC 39.7% #8 of 12 Official leaderboard
MCPMark 17.3% #19 of 28 Official leaderboard
tau-bench 57.4% #16 of 23 Model card
Elimination Game 3 #44 of 56 Official leaderboard
tau2-bench 56.9% #8 of 10 Official leaderboard
Toolathlon 14.8% #44 of 50 Official leaderboard
VitaBench 2.0 21% #18 of 20 Official leaderboard
Mystery Game Puzzles 5.00% #77 of 80 Epoch AI
BrowseComp 28.3% #75 of 77 Model card

Professional work

BenchmarkScoreRankSource
MedHELM 69.7 #2 of 13 Official leaderboard
CadEval 62% #3 of 15 Epoch AI
TaxEval v2 74.8% #29 of 129 Official leaderboard
xbench-Recruitment 61.4 #3 of 8 Official leaderboard
Vals Finance Agent v1 35.9% #21 of 47 Official leaderboard
FailureSensorIQ 64.8% #11 of 24 Official leaderboard
MortgageTax 64.8% #43 of 86 Official leaderboard
ProfBench 47.5% #37 of 72 Official leaderboard
CorpFin v2 59% #76 of 119 Official leaderboard
LegalBench 79.2% #92 of 138 Official leaderboard
GDPval 25.3% #8 of 11 Epoch AI
MedCode 33.8% #77 of 96 Official leaderboard
PRBench 38.1% #37 of 44 Official leaderboard
xbench-Marketing 43.5 #6 of 7 Official leaderboard
MedScribe 69.1% #87 of 98 Official leaderboard

Vision, audio and robotics

BenchmarkScoreRankSource
VisualPuzzles 57% #1 of 29 Official leaderboard
MMMU 81.6% #5 of 136 Official leaderboard
VISTA 51.8% #5 of 54 Official leaderboard
VHELM 76.0 #4 of 41 Official leaderboard
VPCT 57.5% #6 of 28 Epoch AI
MathVista 84.3% #17 of 64 Model card
EnigmaEval 9.21% #15 of 41 Epoch AI
MATH-Vision (MathVision) 58% #40 of 105 Official leaderboard
ZeroBench 11% #27 of 66 Official leaderboard
Vision Arena 1,201 #77 of 136 Official leaderboard
GeoBench 64% #32 of 48 Official leaderboard
CharXiv 72% #48 of 67 Model card
ERQA 30.7% #16 of 17 Official leaderboard

Safety and security

BenchmarkScoreRankSource
Multimodal Troubleshooting Virology 58% #3 of 26 Model card
HELM Safety 97.3 #10 of 75 Official leaderboard
SimpleSafetyTests 100% #12 of 75 Official leaderboard
Capture the Flag (Collegiate) 48% #3 of 13 Model card
AIR-Bench 2024 78.5% #26 of 82 Official leaderboard
Capture the Flag (High School) 80% #3 of 9 Model card
Apollo Research sabotage evaluations 0.17 #1 of 3 Model card
StrongREJECT 96% #8 of 22 Model card
Spiral-Bench v1.2 53.6 #8 of 21 Official leaderboard
Spiral-Bench v1.0 73.3 #5 of 13 Official leaderboard
Virology Capabilities Test (VCT) 37% #3 of 7 Official leaderboard
FORTRESS 21.5% #29 of 66 Official leaderboard
Human Sourced Jailbreaks 99% #4 of 8 Model card
Capture the Flag (Professional) 41% #11 of 21 Model card
Tacit Knowledge and Troubleshooting 68% #20 of 29 Model card
First-Person Fairness evaluation 0.70% #7 of 10 Model card
Anthropic Red Team 98.2% #53 of 75 Official leaderboard
Challenging Refusal Evaluation 90% #7 of 9 Model card
ProtocolQA Open-Ended 27% #20 of 25 Model card
Cyber Range 0% #9 of 10 Model card
CyberGym 2.46% #30 of 33 Official leaderboard
Image generation refusals 98% #2 of 2 Model card

Other

BenchmarkScoreRankSource
HELM Capabilities 81.2 #2 of 60 Official leaderboard
CAIS Risk Index 35.9 #8 of 65 Official leaderboard
ACCoRD 63.8 #77 of 150 Official leaderboard
Conceptual Reasoning Index 41.2 #95 of 149 Official leaderboard
LMCA 26.5% #92 of 139 Epoch AI
SAGE (Vals) 41.1% #57 of 81 Official leaderboard
CAIS Text Capabilities Index 15.4 #57 of 66 Official leaderboard
CAIS Vision Capabilities Index 36.8 #49 of 53 Official leaderboard

OpenAI's models over time

OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.

50 75 100 125 150 175 2023 2024 2025 2026 2027 GPT-6.1 Terra GPT-6 Luna GPT-5.5 Instant GPT-5.4 Mini GPT-5.2 Pro GPT-5 GPT-5 nano 50 75 100 125 150 175 2023 2024 2025 2026 2027 GPT-6.1 Terra GPT-6 Luna

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Introducing OpenAI o3 and o4-mini (OpenAI) 16 Apr 2025

Its system card

Every result in the longer documents the lab published with it. 33 of them aren't in the launch post.

OpenAI o3 and o4-mini System Card 16 Apr 2025

Capabilities

Hallucination

Safety

Robustness

Bias

Alignment

Bio and chem risk

Cyber risk

AI R&D risk

Autonomy