GPT-4 Jun 2023
GPT-4 ranks first on 2 of the 37 benchmarks it has scores on.
OpenAI Released 13 Jun 2023
38
Benchmarks with a score
2
Ranked first, out of five or more models
4
In the top three
OpenAI Jun 2023
GPT-4
Jun 2023 snapshot Closed weights
123.0
#186 of 271
Epoch Capabilities Index
- GPQA Diamond 30.6% #197 /219
- SWT-bench 18.5% #4 /9
- Vectara Hallucination Leaderboard (HHEM) 1.80% #130 /148
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| EQ-Bench v2 | 84.8 | #6 of 85 | Official leaderboard | |
| MMLU | 82.4% | #16 of 127 | Epoch AI | |
| TriviaQA | 84.8% | #7 of 38 | Epoch AI | |
| BIG-Bench Hard | 75.1% | #9 of 45 | Epoch AI | |
| AlpacaEval | 93.8% | #6 of 19 | Official leaderboard | |
| Chinese SimpleQA | 47.4% | #9 of 27 | Official leaderboard | |
| AlpacaEval 2.0 | 30.2% | #16 of 40 | Official leaderboard | |
| LMArena (Elo) | 1,276 | #248 of 333 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 1.80% | #130 of 148 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 90% | #7 of 82 | Epoch AI | |
| ForecastBench | 58.8 | #20 of 86 | Official leaderboard | |
| DTBench | 37.8% | #130 of 178 | Official leaderboard | |
| MATH Level 5 | 23% | #73 of 94 | Epoch AI | |
| GPQA Diamond | 30.6% | #197 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 1.11% | #192 of 202 | Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CodeEditorBench | 85.5% | #1 of 7 | Official leaderboard | |
| Spider | 86.6% | #1 of 5 | Official leaderboard | |
| BigCodeBench | 51.6% | #15 of 69 | Official leaderboard | |
| CRUXEval | 77.1% | #3 of 11 | Official leaderboard | |
| Aider-Edit | 67.7% | #15 of 37 | Official leaderboard | |
| SWT-bench | 18.5% | #4 of 9 | Official leaderboard | |
| BigCodeBench-Hard | 17.6% | #65 of 96 | Official leaderboard | |
| Bird-SQL | 54.9% | #9 of 13 | Official leaderboard | |
| SWE-bench Lite | 23.7% | #11 of 15 | Official leaderboard | |
| WeirdML | 12.4% | #108 of 112 | Official leaderboard | |
| TasteVal | 2.72× | #20 of 20 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GAIA | 64.5% | #23 of 57 | Official leaderboard | |
| LLM Chess | -120 | #78 of 124 | Official leaderboard | |
| ToolComp | 51.4% | #23 of 32 | Official leaderboard | |
| Mystery Game Puzzles | 12% | #62 of 80 | Epoch AI |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PubMedQA | 82% | #1 of 3 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SuperCLUE-Safety | 87.4 | #2 of 7 | Official leaderboard | |
| Situational Awareness Dataset (SAD) | 46.7% | #9 of 17 | Official leaderboard | |
| AIR-Bench 2024 | 64.2% | #45 of 82 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 86.7 | #6 of 71 | Official leaderboard | |
| LMCA | 17.1% | #112 of 139 | Epoch AI | |
| Conceptual Reasoning Index | 25.3 | #123 of 149 | Official leaderboard | |
| ACCoRD | 37.3 | #134 of 150 | Official leaderboard |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.