GPT-4o Aug 2024
GPT-4o ranks first on Image2Struct.
OpenAI Released 6 Aug 2024
79
Benchmarks with a score
1
Ranked first, out of five or more models
13
In the top three
OpenAI Aug 2024
GPT-4o
Aug 2024 snapshot Closed weights
128.8
#167 of 271
Epoch Capabilities Index
- GPQA Diamond 49.2% #146 /219
- FrontierMath 0.34% #65 /70
- Vals Finance Agent 8.06% #46 /49
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Chinese SimpleQA | 59.7% | #2 of 27 | Official leaderboard | |
| MMLU | 84.3% | #13 of 127 | Epoch AI | |
| QuALITY | 83.1% | #1 of 4 | Official leaderboard | |
| RewardBench | 86.7% | #14 of 48 | Official leaderboard | |
| MMBench-CN v1.1 | 79.2% | #55 of 178 | Official leaderboard | |
| LongBench v2 | 51.2% | #11 of 33 | Official leaderboard | |
| MMLU-Pro | 74.7% | #74 of 212 | Official leaderboard | |
| LLM Confabulation Leaderboard | 15.3% | #22 of 58 | Official leaderboard | |
| SEAL Multilingual | 1,067 | #8 of 19 | Official leaderboard | |
| Vectara Hallucination Leaderboard (2025 dataset) | 9.60% | #55 of 104 | Official leaderboard | |
| SuperGPQA | 41.6% | #33 of 58 | Official leaderboard | |
| LMArena (Elo) | 1,336 | #191 of 333 | Official leaderboard | |
| RewardBench 2 | 64.9% | #25 of 42 | Official leaderboard | |
| BullshitBench | 16.4% | #76 of 107 | Official leaderboard | |
| SEAL Precise Instruction Following | 80.2 | #14 of 18 | Official leaderboard | |
| SimpleQA Verified | 26% | #67 of 84 | Epoch AI | |
| BullshitBench V2 | 12% | #102 of 122 | Official leaderboard | |
| MultiNRC | 12.4% | #35 of 38 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM1k | 95.7% | #2 of 15 | Official leaderboard | |
| ZebraLogic | 31.7% | #10 of 46 | Official leaderboard | |
| MATH Level 5 | 53.3% | #47 of 94 | Epoch AI | |
| U-MATH | 50.2% | #16 of 30 | Official leaderboard | |
| GPQA Diamond | 49.2% | #146 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 6.39% | #166 of 202 | Epoch AI | |
| FrontierMath | 0.34% | #65 of 70 | Epoch AI | |
| SimpleBench | 17.8% | #97 of 100 | Official leaderboard | |
| FrontierMath Tiers 1-3 v2 | 0.35% | #88 of 89 | Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval+ | 87.2% | #3 of 46 | Official leaderboard | |
| HumanEval-V | 27.7% | #3 of 24 | Official leaderboard | |
| BigCodeBench-Hard | 30.8% | #17 of 96 | Official leaderboard | |
| Aider-Edit | 71.4% | #9 of 37 | Official leaderboard | |
| MBPP+ | 72.2% | #12 of 35 | Official leaderboard | |
| SEAL Coding | 1,045 | #10 of 26 | Official leaderboard | |
| SWE-bench Lite | 32% | #8 of 15 | Official leaderboard | |
| SWE-bench Verified | 38.8% | #44 of 60 | Official leaderboard | |
| Aider Polyglot | 23.1% | #40 of 54 | Epoch AI | |
| LiveCodeBench v6 | 29.5% | #17 of 21 | Official leaderboard | |
| MLE-bench | 8.63% | #10 of 12 | Official leaderboard | |
| LiveCodeBench v5 | 29.5% | #18 of 21 | Official leaderboard | |
| SWE-bench Multimodal | 12.2% | #9 of 9 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| VideoGameBench | 0.09% | #2 of 5 | Official leaderboard | |
| ToolComp | 59.9% | #16 of 32 | Official leaderboard | |
| Factorio Learning Environment | 87599.0 | #3 of 6 | Official leaderboard | |
| LLM Chess | -134 | #83 of 124 | Official leaderboard | |
| Step Race | 1 | #50 of 59 | Official leaderboard | |
| Chess Puzzles | 6.00% | #78 of 83 | Epoch AI | |
| MCP-Universe | 15.6% | #29 of 30 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedXpertQA-MM | 42.8% | #2 of 9 | Official leaderboard | |
| TaxEval v2 | 74.5% | #35 of 129 | Official leaderboard | |
| CaseLaw v2 | 59.7% | #25 of 53 | Official leaderboard | |
| LegalBench | 82.2% | #72 of 138 | Official leaderboard | |
| MortgageTax | 61% | #60 of 86 | Official leaderboard | |
| Vals Finance Agent v1 | 19.3% | #34 of 47 | Official leaderboard | |
| CadEval | 26% | #12 of 15 | Epoch AI | |
| CorpFin v2 | 45.9% | #103 of 119 | Official leaderboard | |
| Vals Finance Agent | 8.06% | #46 of 49 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Image2Struct | 94.7 | #1 of 18 | Official leaderboard | |
| A-Bench | 79.1% | #8 of 116 | Official leaderboard | |
| TOMATO | 37.7% | #2 of 22 | Official leaderboard | |
| MMVU | 67.4% | #3 of 28 | Official leaderboard | |
| Video-MME | 71.9% | #6 of 50 | Epoch AI | |
| MME-CoT | 64% | #2 of 11 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 66.5% | #28 of 152 | Official leaderboard | |
| MMStar | 64.7% | #35 of 173 | Official leaderboard | |
| MMBench-EN v1.1 | 81.8% | #39 of 178 | Official leaderboard | |
| MMBench | 82.1% | #38 of 155 | Official leaderboard | |
| Video-MMMU | 61.2% | #6 of 23 | Official leaderboard | |
| VHELM | 59.8 | #15 of 41 | Official leaderboard | |
| VisualPuzzles | 41.3% | #13 of 29 | Official leaderboard | |
| VideoDR (with search) | 43% | #3 of 6 | Official leaderboard | |
| VISTA | 34.9% | #40 of 54 | Official leaderboard | |
| Vision Arena | 1,118 | #113 of 136 | Official leaderboard | |
| MADQA | 48.6% | #14 of 16 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AIR-Bench 2024 | 62.4% | #48 of 82 | Official leaderboard | |
| FORTRESS | 47.2% | #49 of 66 | Official leaderboard | |
| Human Pathogen Capabilities Test v2 (HPCT v2) | 30% | #65 of 71 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 92.8 | #2 of 71 | Official leaderboard | |
| OpenVLM Leaderboard | 71.6 | #27 of 173 | Official leaderboard | |
| LiveBench | 55.3% | #18 of 48 | Epoch AI |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.