GPT-4o Nov 2024
GPT-4o ranks first on 3 of the 67 benchmarks it has scores on.
OpenAI Released 20 Nov 2024
69
Benchmarks with a score
3
Ranked first, out of five or more models
11
In the top three
OpenAI Nov 2024
GPT-4o
Nov 2024 snapshot Closed weights
128.7
#169 of 271
Epoch Capabilities Index
- Humanity's Last Exam 2.72% #47 /47
- SWE-bench Verified 31% #32 /32
- CritPt 0% #118 /160
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMLU | 88.1% | #1 of 127 | Epoch AI | |
| MMLU-CF | 73.4% | #1 of 37 | Official leaderboard | |
| Creation-MMBench | 0.00 | #2 of 19 | Official leaderboard | |
| MMBench-CN v1.1 | 83.9% | #21 of 178 | Official leaderboard | |
| ToRR | 49.8 | #2 of 14 | Official leaderboard | |
| AgentIF | 58.5% | #2 of 12 | Official leaderboard | |
| Lech Mazur Creative Writing | 8.18 | #13 of 44 | Epoch AI | |
| LongBench v2 | 51.4% | #10 of 33 | Official leaderboard | |
| MMLU-Pro | 77.9% | #67 of 212 | Official leaderboard | |
| FaithJudge | 15.8% | #13 of 37 | Official leaderboard | |
| SuperGPQA | 44.4% | #30 of 58 | Official leaderboard | |
| LLM Confabulation Leaderboard | 17.2% | #31 of 58 | Official leaderboard | |
| SEAL Precise Instruction Following | 82.5 | #10 of 18 | Official leaderboard | |
| Humanity's Last Exam | 2.72% | #47 of 47 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ForecastBench | 59.6 | #23 of 86 | Official leaderboard | |
| miniCTX-v2 | 27.8% | #2 of 4 | Official leaderboard | |
| MATH Level 5 | 49.8% | #53 of 94 | Epoch AI | |
| GPQA Diamond | 47.9% | #153 of 219 | Epoch AI | |
| Misguided Attention | 33.7% | #32 of 45 | Official leaderboard | |
| CritPt | 0% | #118 of 160 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 6.25% | #168 of 202 | Epoch AI | |
| ARC-AGI-2 | 0% | #88 of 95 | Official leaderboard | |
| ARC-AGI | 4.50% | #88 of 92 | Official leaderboard | |
| FrontierMath | 0.34% | #68 of 70 | Epoch AI | |
| PolyMath | 13.7% | #37 of 38 | Official leaderboard | |
| AMO-Bench | 1.50% | #21 of 21 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench | 53.5% | #5 of 69 | Official leaderboard | |
| SEAL Coding | 1,132 | #3 of 26 | Official leaderboard | |
| BigCodeBench-Hard | 31.1% | #15 of 96 | Official leaderboard | |
| Aider-Edit | 71.4% | #10 of 37 | Official leaderboard | |
| BIRD-CRITIC | 27.5% | #3 of 9 | Official leaderboard | |
| AutoCodeBench | 41.1% | #21 of 34 | Official leaderboard | |
| Aider Polyglot | 18.2% | #43 of 54 | Epoch AI | |
| Multi-SWE-bench | 3.80% | #9 of 11 | Official leaderboard | |
| WeirdML | 25.1% | #100 of 112 | Official leaderboard | |
| SWE-bench Verified | 31% | #32 of 32 | Epoch AI | |
| GSO-Bench | 0% | #30 of 30 | Epoch AI |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | 71.7% | #4 of 67 | Official leaderboard | |
| AceBench | 85.4% | #1 of 16 | Official leaderboard | |
| CRMArena | 54.4% | #2 of 9 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.90 | #9 of 31 | Official leaderboard | |
| AndroidLab | 31.2% | #1 of 2 | Official leaderboard | |
| LLM Chess | -108 | #75 of 124 | Official leaderboard | |
| AgentBench FC | 39.6% | #9 of 12 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TheAgentCompany | 8.60% | #8 of 14 | Epoch AI | |
| GDPval | 9.90% | #11 of 11 | Epoch AI |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-EN v1.1 | 84.8% | #19 of 178 | Official leaderboard | |
| KRETA | 84.6% | #2 of 17 | Official leaderboard | |
| CharXiv | 50.5% | #10 of 77 | Official leaderboard | |
| Video-MME | 71.9% | #7 of 50 | Epoch AI | |
| MMStar | 65.1% | #32 of 173 | Official leaderboard | |
| MMMU | 70.7% | #30 of 136 | Official leaderboard | |
| LVBench | 48.9% | #2 of 9 | Official leaderboard | |
| VHELM | 62.7 | #13 of 41 | Official leaderboard | |
| GeoBench | 71% | #14 of 27 | Epoch AI | |
| VPCT | 40% | #15 of 28 | Epoch AI | |
| VISTA | 38% | #38 of 54 | Official leaderboard | |
| EnigmaEval | 0.80% | #37 of 41 | Epoch AI | |
| Drone-Bench | 20.4% | #18 of 19 | Official leaderboard | |
| Math-VR | 4.30% | #25 of 26 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Cybench | 12.5% | #14 of 21 | Epoch AI | |
| Human Pathogen Capabilities Test v2 (HPCT v2) | 31.6% | #62 of 71 | Official leaderboard | |
| PCT (Polarized Contrastive Pairs) | 59.6% | #43 of 46 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 72.0 | #25 of 173 | Official leaderboard | |
| LiveBench | 52.2% | #24 of 48 | Epoch AI | |
| HELM Capabilities | 63.4 | #36 of 60 | Official leaderboard | |
| CAIS Vision Capabilities Index | 36.0 | #50 of 53 | Official leaderboard | |
| CAIS Risk Index | 74.4 | #64 of 65 | Official leaderboard | |
| CAIS Text Capabilities Index | 7.90 | #65 of 66 | Official leaderboard |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.