o1-mini ranks first on 5 of the 67 benchmarks it has scores on.
Closed weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AgentIF | 59.8% | #1 of 12 | Official leaderboard | |
| MMLU-Pro | 80.3% | #56 of 212 | Official leaderboard | |
| Chinese SimpleQA | 44.1% | #11 of 27 | Official leaderboard | |
| SuperGPQA | 45.2% | #27 of 58 | Official leaderboard | |
| BirthdayFacts | 24% | #1 of 2 | Model card | |
| EQ-Bench v2 | 72.0 | #48 of 85 | Official leaderboard | |
| BuzzBench | 46.2 | #13 of 23 | Official leaderboard | |
| LMArena (Elo) | 1,337 | #189 of 333 | Official leaderboard | |
| LLM Confabulation Leaderboard | 18.6% | #35 of 58 | Official leaderboard | |
| LongBench v2 | 38.9% | #24 of 33 | Official leaderboard | |
| SimpleQA Hallucination Rate | 60% | #9 of 12 | Model card | |
| ECLeKTic | 8.69% | #28 of 35 | Official leaderboard | |
| SimpleQA | 7.00% | #52 of 59 | Model card | |
| FACTS Grounding | 62.1% | #30 of 34 | Official leaderboard | |
| Lech Mazur Creative Writing | 6.49 | #39 of 44 | Epoch AI | |
| Vectara Hallucination Leaderboard (HHEM) | 1.40% | #138 of 148 | Official leaderboard | |
| Open Ended Questions (hallucination) | 0.93 | #2 of 2 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| KOR-Bench | 70.2% | #2 of 33 | Official leaderboard | |
| ZebraLogic | 52.6% | #5 of 46 | Official leaderboard | |
| Omni-MATH | 60.5% | #1 of 9 | Official leaderboard | |
| U-MATH | 76.3% | #5 of 30 | Official leaderboard | |
| MATH Level 5 | 89.2% | #18 of 94 | Epoch AI | |
| PolyMath | 36.6% | #15 of 38 | Official leaderboard | |
| GPQA Diamond | 62.4% | #124 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 46.9% | #127 of 202 | Epoch AI | |
| QuantBench | 48% | #2 of 3 | Model card | |
| FrontierMath | 1.72% | #55 of 70 | Epoch AI | |
| ARC-AGI-2 | 0.83% | #82 of 95 | Official leaderboard | |
| ARC-AGI | 14% | #81 of 92 | Official leaderboard | |
| SimpleBench | 18.1% | #96 of 100 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval+ | 89% | #1 of 46 | Official leaderboard | |
| SEAL Coding | 1,237 | #1 of 26 | Official leaderboard | |
| MBPP+ | 78.8% | #2 of 35 | Official leaderboard | |
| MHPP | 66.2% | #2 of 23 | Official leaderboard | |
| BigCodeBench-Hard | 27.4% | #29 of 96 | Official leaderboard | |
| BigO(Bench) | 35.6% | #8 of 18 | Official leaderboard | |
| BIRD-CRITIC | 25% | #4 of 9 | Official leaderboard | |
| StructEval | 75.6% | #13 of 28 | Official leaderboard | |
| Aider-Edit | 61.1% | #21 of 37 | Official leaderboard | |
| Aider Polyglot | 32.9% | #36 of 54 | Epoch AI | |
| OpenAI Research Engineer Interview: Multiple Choice | 77.3% | #7 of 9 | Model card | |
| WeirdML | 36.3% | #92 of 112 | Official leaderboard | |
| SciCode | 1.50% | #15 of 18 | Official leaderboard | |
| SWE-bench Verified | 35.8% | #144 of 150 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Step Race | 2 | #28 of 59 | Official leaderboard | |
| LLM Chess | -52 | #68 of 124 | Official leaderboard | |
| LisanBench | 295.3 | #81 of 133 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Situational Awareness Dataset (SAD) | 55.4% | #3 of 17 | Official leaderboard | |
| Jailbreak Augmented Examples | 100% | #1 of 5 | Model card | |
| Challenging Refusal Evaluation | 93.2% | #2 of 9 | Model card | |
| HELM Safety | 95.5 | #20 of 75 | Official leaderboard | |
| MakeMeSay | 38.9% | #5 of 9 | Model card | |
| StrongREJECT | 83% | #13 of 22 | Model card | |
| Production Jailbreaks | 99% | #3 of 5 | Model card | |
| Standard Refusal Evaluation | 99% | #5 of 8 | Model card | |
| Anthropic Red Team | 98.3% | #51 of 75 | Official leaderboard | |
| SimpleSafetyTests | 97% | #55 of 75 | Official leaderboard | |
| Political Persuasion Parallel Generation | 46.8% | #3 of 4 | Model card | |
| Tacit Knowledge and Troubleshooting | 66.7% | #22 of 29 | Model card | |
| Cybench | 10% | #19 of 24 | Official leaderboard | |
| XSTest | 94.8% | #13 of 16 | Model card | |
| AIR-Bench 2024 | 45.3% | #69 of 82 | Official leaderboard | |
| Capture the Flag (Professional) | 3.90% | #18 of 21 | Model card | |
| Human Sourced Jailbreaks | 95% | #7 of 8 | Model card | |
| Capture the Flag (High School) | 28.7% | #8 of 9 | Model card | |
| LAB-Bench CloningScenarios | 39.4% | #10 of 11 | Model card | |
| Capture the Flag (Collegiate) | 0% | #12 of 13 | Model card | |
| LAB-Bench ProtocolQA | 67.6% | #14 of 15 | Model card | |
| MakeMePay | 0.70% | #6 of 6 | Model card | |
| ChangeMyView | 69.5% | #5 of 5 | Model card | |
| WildChat (toxic conversations) | 95.7% | #4 of 4 | Model card | |
| External red team jailbreaks | 15% | #2 of 2 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LiveBench | 57.8% | #17 of 48 | Epoch AI |
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Every result in the longer documents the lab published with it. 37 of them aren't in the launch post.