OpenAI o3-mini 31 Jan 2025
o3-mini
o3-mini ranks first on 3 of the 111 benchmarks it has scores on.
117
Benchmarks with a score
3
Ranked first, out of five or more models
12
In the top three
OpenAI Jan 2025
o3-mini
Closed weights Small
140.3
#121 of 271
Epoch Capabilities Index
- CritPt 0.29% #106 /160
- GPQA Diamond 77% #94 /219
- FrontierMath 12.4% #36 /70
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BIG-Bench Extra Hard | 54.2% | #1 of 11 | Official leaderboard | |
| MGSM | 92% | #5 of 52 | Model card | |
| FaithJudge | 11.6% | #6 of 37 | Official leaderboard | |
| Arena-Hard v2.0 | 66.1% | #5 of 23 | Official leaderboard | |
| MMLU-Pro | 79.4% | #60 of 212 | Official leaderboard | |
| SuperGPQA | 55.2% | #18 of 58 | Official leaderboard | |
| SEAL Multilingual | 1,079 | #6 of 19 | Official leaderboard | |
| MMLU | 86.9% | #42 of 127 | Model card | |
| LMArena (Elo) | 1,364 | #165 of 333 | Official leaderboard | |
| FACTS Grounding | 77.6% | #18 of 34 | Official leaderboard | |
| LLM Confabulation Leaderboard | 17.9% | #33 of 58 | Official leaderboard | |
| WritingBench | 68.0 | #27 of 45 | Official leaderboard | |
| ECLeKTic | 14.5% | #21 of 35 | Official leaderboard | |
| Fiction.LiveBench | 50% | #41 of 57 | Epoch AI | |
| SimpleQA | 13.8% | #43 of 59 | Model card | |
| Global MMLU Lite | 78% | #35 of 43 | Official leaderboard | |
| PersonQA | 21.7% | #8 of 9 | Model card | |
| SimpleQA Verified | 15.3% | #76 of 84 | Epoch AI | |
| Lech Mazur Creative Writing | 6.17 | #43 of 44 | Epoch AI | |
| Vectara Hallucination Leaderboard (HHEM) | 0.80% | #146 of 148 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ZebraLogic | 91.7% | #1 of 46 | Official leaderboard | |
| MATH Level 5 | 96.5% | #8 of 94 | Epoch AI | |
| U-MATH | 82.2% | #3 of 30 | Official leaderboard | |
| MATH-500 | 97.9% | #10 of 67 | Model card | |
| AIME 2024 | 87.3% | #26 of 87 | Model card | |
| PolyMath | 38.6% | #13 of 38 | Official leaderboard | |
| ForecastBench | 61.6 | #30 of 86 | Official leaderboard | |
| GPQA Diamond | 77% | #94 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 76.9% | #92 of 202 | Epoch AI | |
| AMO-Bench | 32.3% | #10 of 21 | Official leaderboard | |
| Misguided Attention | 46.8% | #22 of 45 | Official leaderboard | |
| AIME 2025 | 86.7% | #26 of 53 | Official leaderboard | |
| FrontierMath Tier 4 | 4.17% | #27 of 55 | Epoch AI | |
| PHYBench | 25% | #14 of 28 | Official leaderboard | |
| FrontierMath | 12.4% | #36 of 70 | Epoch AI | |
| HMMT 2025 | 67.5% | #32 of 53 | Official leaderboard | |
| DTBench | 48% | #113 of 178 | Official leaderboard | |
| CritPt | 0.29% | #106 of 160 | Epoch AI | |
| ARC-AGI-2 | 2.99% | #72 of 95 | Official leaderboard | |
| ARC-AGI | 34.5% | #72 of 92 | Official leaderboard | |
| PutnamBench | 0% | #11 of 14 | Official leaderboard | |
| SimpleBench | 22.8% | #89 of 100 | Official leaderboard | |
| FrontierMath Tiers 1-3 v2 | 18.6% | #80 of 89 | Epoch AI | |
| General365 | 44.4% | #21 of 23 | Official leaderboard | |
| FrontierMath Tier 4 v2 | 0% | #69 of 71 | Epoch AI | |
| USAMO 2025 | 2.08% | #10 of 10 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench-Hard | 35.5% | #3 of 96 | Official leaderboard | |
| SciCode | 10.8% | #1 of 18 | Official leaderboard | |
| SEAL Coding | 1,137 | #2 of 26 | Official leaderboard | |
| OpenAI Research Engineer Interview: Multiple Choice | 80% | #2 of 9 | Model card | |
| Aider Polyglot | 60.4% | #16 of 54 | Epoch AI | |
| LiveCodeBench v6 | 67.4% | #8 of 21 | Official leaderboard | |
| Codeforces rating | 2,130 | #16 of 41 | Model card | |
| LiveCodeBench v5 | 69.3% | #9 of 21 | Official leaderboard | |
| SWE-bench Lite | 32.3% | #7 of 15 | Official leaderboard | |
| WeirdML | 43.7% | #64 of 112 | Official leaderboard | |
| LiveSQLBench | 23.7% | #21 of 36 | Official leaderboard | |
| SWE-bench Verified | 42.4% | #37 of 60 | Official leaderboard | |
| Multi-SWE-bench | 5.27% | #7 of 11 | Official leaderboard | |
| OpenAI Research Engineer Interview: Coding | 92% | #5 of 7 | Model card | |
| MLE-bench | 11% | #13 of 18 | Model card | |
| Agentic tasks (autonomous replication and adaptation) | 27% | #3 of 4 | Model card | |
| GSO-Bench | 1.30% | #29 of 30 | Official leaderboard | |
| OpenAI PRs | 0% | #12 of 12 | Model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Step Race | 4 | #11 of 59 | Official leaderboard | |
| ToolComp | 65.3% | #6 of 32 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 64.6% | #16 of 67 | Official leaderboard | |
| SnakeBench | 28 | #46 of 183 | Official leaderboard | |
| LisanBench | 1366.3 | #50 of 133 | Official leaderboard | |
| LLM Chess | 403 | #47 of 124 | Official leaderboard | |
| Elimination Game | 4 | #25 of 56 | Official leaderboard | |
| AgentBench FC | 40.9% | #6 of 12 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.85 | #19 of 31 | Official leaderboard | |
| Chess Puzzles | 15% | #67 of 83 | Epoch AI | |
| Mystery Game Puzzles | 7.00% | #72 of 80 | Epoch AI |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedHELM | 57.2 | #4 of 13 | Official leaderboard | |
| CadEval | 54% | #5 of 15 | Epoch AI | |
| LEXam | 48.1% | #14 of 31 | Official leaderboard | |
| TaxEval v2 | 69.4% | #90 of 129 | Official leaderboard | |
| LegalBench | 71.5% | #111 of 138 | Official leaderboard | |
| Vals Finance Agent v1 | 12.7% | #38 of 47 | Official leaderboard | |
| CorpFin v2 | 45.3% | #105 of 119 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BaxBench | 45.9% | #7 of 34 | Official leaderboard | |
| MakeMeSay | 62% | #2 of 9 | Model card | |
| HELM Safety | 96.2 | #18 of 75 | Official leaderboard | |
| BioLP-bench | 41% | #3 of 9 | Model card | |
| AIR-Bench 2024 | 74.9% | #28 of 82 | Official leaderboard | |
| Standard Refusal Evaluation | 100% | #3 of 8 | Model card | |
| Production Jailbreaks | 100% | #2 of 5 | Model card | |
| Capture the Flag (High School) | 61% | #4 of 9 | Model card | |
| Multimodal Troubleshooting Virology | 53% | #12 of 26 | Model card | |
| SimpleSafetyTests | 99% | #35 of 75 | Official leaderboard | |
| Capture the Flag (Collegiate) | 21% | #7 of 13 | Model card | |
| Cybench | 22.5% | #13 of 24 | Official leaderboard | |
| XSTest | 88% | #9 of 16 | Model card | |
| ChangeMyView | 82.9% | #3 of 5 | Model card | |
| FORTRESS | 30.1% | #40 of 66 | Official leaderboard | |
| Anthropic Red Team | 98.6% | #46 of 75 | Official leaderboard | |
| Human Sourced Jailbreaks | 97% | #5 of 8 | Model card | |
| Capture the Flag (Professional) | 21% | #14 of 21 | Model card | |
| Challenging Refusal Evaluation | 90% | #6 of 9 | Model card | |
| StrongREJECT | 73% | #15 of 22 | Model card | |
| Contextual Nuclear Knowledge | 73% | #3 of 4 | Model card | |
| MakeMePay | 1.00% | #5 of 6 | Model card | |
| ProtocolQA Open-Ended | 23% | #23 of 25 | Model card | |
| Tacit Knowledge and Troubleshooting | 58% | #28 of 29 | Model card | |
| Jailbreak Augmented Examples | 100% | #5 of 5 | Model card | |
| Political Persuasion Parallel Generation | 35.5% | #4 of 4 | Model card | |
| Radiological and Nuclear Expert Knowledge | 60% | #4 of 4 | Model card | |
| Tacit Knowledge Brainstorm (Open-Ended) | 51% | #2 of 2 | Model card | |
| Gray Swan Jailbreak Arena | 3.60% | #1 of 1 | Model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LiveBench | 75.9% | #4 of 48 | Epoch AI | |
| ACCoRD | 56.5 | #99 of 150 | Official leaderboard | |
| Conceptual Reasoning Index | 32.3 | #110 of 149 | Official leaderboard | |
| LMCA | 19% | #107 of 139 | Epoch AI | |
| CAIS Text Capabilities Index | 12.7 | #59 of 66 | Official leaderboard | |
| CAIS Risk Index | 67.6 | #61 of 65 | Official leaderboard |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 37 of them aren't in the launch post.
OpenAI o3-mini System Card 31 Jan 2025
Capabilities
Hallucination
- PersonQA 21.7
Safety
Robustness
Bio and chem risk
- Long-form Biological Risk Questions
- Expert comparisons on biothreat information
- Biotool and Wet Lab Actions
- Multimodal Troubleshooting Virology 53
- BioLP-bench 41
- ProtocolQA Open-Ended 23
- Tacit Knowledge and Troubleshooting 58
- Tacit Knowledge Brainstorm (Open-Ended) 51
- Structured expert probing campaign: radiological and nuclear
- Contextual Nuclear Knowledge 73
- Radiological and Nuclear Expert Knowledge 60
Cyber risk
Autonomy
Persuasion
- ChangeMyView 82.9
- Political Persuasion Parallel Generation 35.5
- MakeMePay 1
- MakeMeSay 62