Introducing GPT-4.5 27 Feb 2025
GPT-4.5
GPT-4.5 ranks first on 2 of the 75 benchmarks it has scores on.
84
Benchmarks with a score
2
Ranked first, out of five or more models
5
In the top three
OpenAI Feb 2025
GPT-4.5
Closed weights Flagship
136.7
#140 of 271
Epoch Capabilities Index
- Humanity's Last Exam 5.44% #40 /47
- ZeroBench 6.00% #35 /66
- GPQA Diamond 68.7% #111 /219
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU-Pro | 86.1% | #24 of 212 | Official leaderboard | |
| MMBench-CN v1.1 | 83.3% | #25 of 178 | Official leaderboard | |
| SimpleQA | 62.5% | #9 of 59 | Model card | |
| FaithJudge | 11.9% | #7 of 37 | Official leaderboard | |
| LMArena (Elo) | 1,445 | #69 of 333 | Official leaderboard | |
| Creation-MMBench | -5.88 | #5 of 19 | Official leaderboard | |
| LLM Confabulation Leaderboard | 13.6% | #16 of 58 | Official leaderboard | |
| SimpleQA Hallucination Rate | 37.1% | #4 of 12 | Model card | |
| PersonQA | 78% | #3 of 9 | Model card | |
| Arena-Hard v2.0 | 50% | #10 of 23 | Official leaderboard | |
| Fiction.LiveBench | 63.9% | #28 of 57 | Epoch AI | |
| MMMLU | 85.1% | #33 of 62 | Model card | |
| Lech Mazur Creative Writing | 7.56 | #27 of 44 | Epoch AI | |
| WritingBench | 67.6 | #28 of 45 | Official leaderboard | |
| Creative Writing v3 | 60.6 | #81 of 120 | Official leaderboard | |
| EQ-Bench 3 | 1,070 | #59 of 78 | Official leaderboard | |
| Humanity's Last Exam | 5.44% | #40 of 47 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 1.20% | #143 of 148 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH Level 5 | 78.6% | #29 of 94 | Epoch AI | |
| GPQA Diamond | 68.7% | #111 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 37.8% | #131 of 202 | Epoch AI | |
| PolyMath | 26.9% | #25 of 38 | Official leaderboard | |
| SimpleBench | 34.5% | #75 of 100 | Official leaderboard | |
| ForecastBench | 64.6 | #68 of 86 | Official leaderboard | |
| ARC-AGI-2 | 0.80% | #84 of 95 | Official leaderboard | |
| AIME 2024 | 36.7% | #77 of 87 | Model card | |
| ARC-AGI | 10.3% | #83 of 92 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenAI Research Engineer Interview: Multiple Choice | 80% | #3 of 9 | Model card | |
| Agentic tasks (autonomous replication and adaptation) | 40% | #2 of 4 | Model card | |
| SWE-Lancer Diamond (SWE Manager) | 44% | #2 of 4 | Model card | |
| Aider Polyglot | 44.9% | #31 of 54 | Epoch AI | |
| WeirdML | 39.4% | #80 of 112 | Official leaderboard | |
| MLE-bench | 11% | #15 of 18 | Model card | |
| OpenAI PRs | 7.00% | #11 of 12 | Model card | |
| SWE-bench Verified | 38% | #143 of 150 | Model card | |
| SWE-Lancer | 32.6% | #10 of 10 | Model card | |
| OpenAI Research Engineer Interview: Coding | 79% | #7 of 7 | Model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | 70.3% | #5 of 67 | Official leaderboard | |
| Elimination Game | 5 | #13 of 56 | Official leaderboard | |
| ToolComp | 63.8% | #10 of 32 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.90 | #10 of 31 | Official leaderboard | |
| LLM Chess | -111 | #77 of 124 | Official leaderboard | |
| Step Race | 1 | #53 of 59 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMStar | 69.3% | #15 of 173 | Official leaderboard | |
| CharXiv | 55.4% | #7 of 77 | Official leaderboard | |
| VHELM | 72.8 | #5 of 41 | Official leaderboard | |
| MMBench-EN v1.1 | 83.5% | #27 of 178 | Official leaderboard | |
| MMMU | 74.4% | #23 of 136 | Official leaderboard | |
| VPCT | 45% | #11 of 28 | Epoch AI | |
| Vision Arena | 1,225 | #65 of 136 | Official leaderboard | |
| MATH-Vision (MathVision) | 47.3% | #55 of 105 | Official leaderboard | |
| ZeroBench | 6.00% | #35 of 66 | Official leaderboard | |
| VISTA | 42.1% | #30 of 54 | Official leaderboard | |
| EnigmaEval | 3.18% | #27 of 41 | Epoch AI |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MakeMeSay | 72% | #1 of 9 | Model card | |
| SimpleSafetyTests | 100% | #9 of 75 | Official leaderboard | |
| Multimodal Troubleshooting Virology | 56% | #4 of 26 | Model card | |
| MakeMePay | 57% | #1 of 6 | Model card | |
| HELM Safety | 96.5 | #15 of 75 | Official leaderboard | |
| WildChat (toxic conversations) | 98% | #1 of 4 | Model card | |
| Anthropic Red Team | 99.4% | #24 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 74.2% | #29 of 82 | Official leaderboard | |
| Human Sourced Jailbreaks | 99% | #3 of 8 | Model card | |
| Tacit Knowledge and Troubleshooting | 72% | #14 of 29 | Model card | |
| XSTest | 85% | #8 of 16 | Model card | |
| Challenging Red Teaming Evaluation 1 (created for o3-mini) | 51% | #1 of 2 | Model card | |
| Image input safety evaluations | 99% | #1 of 2 | Model card | |
| Capture the Flag (High School) | 53% | #5 of 9 | Model card | |
| WMDP Biology | 85% | #4 of 7 | Model card | |
| Cybench | 17.5% | #16 of 24 | Official leaderboard | |
| Long-form Biological Risk Questions | 0% | #2 of 3 | Model card | |
| Capture the Flag (Collegiate) | 16% | #9 of 13 | Model card | |
| Standard Refusal Evaluation | 99% | #6 of 8 | Model card | |
| Radiological and Nuclear Expert Knowledge | 68% | #3 of 4 | Model card | |
| BioLP-bench | 29% | #7 of 9 | Model card | |
| StrongREJECT | 34% | #18 of 22 | Model card | |
| Virology Capabilities Test (VCT) | 28.3% | #6 of 7 | Official leaderboard | |
| Capture the Flag (Professional) | 2.00% | #19 of 21 | Model card | |
| ProtocolQA Open-Ended | 18% | #24 of 25 | Model card | |
| Challenging Refusal Evaluation | 85% | #9 of 9 | Model card | |
| Contextual Nuclear Knowledge | 71% | #4 of 4 | Model card | |
| Challenging Red Teaming Evaluation 2 (created for deep research) | 46% | #2 of 2 | Model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 75.3 | #11 of 173 | Official leaderboard | |
| LiveBench | 69% | #8 of 48 | Epoch AI |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 33 of them aren't in the launch post.
OpenAI GPT-4.5 System Card 27 Feb 2025
Capabilities
Hallucination
- PersonQA 78