- MMLU (5-shot) 86.4
- HellaSwag (10-shot) 95.3
- AI2 Reasoning Challenge (ARC) (25-shot) 96.3
- WinoGrande (5-shot) 87.5
- HumanEval (0-shot) 67
- DROP (F1 score) (3-shot) 80.9
- GSM-8K (5-shot chain-of-thought) 92
- VQAv2 VQA score (test-dev) 0-shot 77.2
- TextVQA VQA score (val) 0-shot 78
- ChartQA Relaxed accuracy (test) 78.5
- AI2 Diagram (AI2D) Accuracy (test) 0-shot 78.2
- DocVQA ANLS score (test) 0-shot (pixel-only) 88.4
- Infographic VQA ANLS score (test) 0-shot (pixel-only) 75.1
- TVQA Accuracy (val) 0-shot 87.3
- LSMDC Fill-in-the-blank accuracy (test) 0-shot 45.7
GPT-4 Mar 2023
GPT-4 ranks first on 2 of the 29 benchmarks it has scores on.
68
Benchmarks with a score
2
Ranked first, out of five or more models
7
In the top three
OpenAI Mar 2023
GPT-4
Mar 2023 snapshot Closed weights Flagship
125.7
#181 of 271
Epoch Capabilities Index
- GPQA Diamond 35.7% #185 /219
- TextVQA 78% #10 /17
- AI2D 78.2% #41 /44
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| EQ-Bench v2 | 85.7 | #3 of 85 | Official leaderboard | |
| HellaSwag | 95.3% | #3 of 47 | Model card | |
| ARC-Challenge | 96.3% | #3 of 45 | Model card | |
| WinoGrande | 87.5% | #4 of 42 | Model card | |
| AlpacaEval | 94.8% | #3 of 19 | Official leaderboard | |
| GRE Verbal | 169.0 | #1 of 6 | Model card | |
| LSAT | 163.0 | #1 of 4 | Model card | |
| AlpacaEval 2.0 | 35.3% | #11 of 40 | Official leaderboard | |
| MMLU | 86.4% | #45 of 127 | Model card | |
| QuALITY | 82.6% | #2 of 4 | Official leaderboard | |
| TruthfulQA | 60% | #15 of 28 | Model card | |
| DROP | 80.9% | #23 of 40 | Model card | |
| LMArena (Elo) | 1,288 | #238 of 333 | Official leaderboard | |
| Creative Writing v3 | 50.8 | #106 of 120 | Official leaderboard | |
| EQ-Bench 3 | 653 | #75 of 78 | Official leaderboard | |
| GRE Analytical Writing | 4.00 | #3 of 3 | Model card | |
| AP Art History | 5.00 | #1 of 1 | Model card | |
| AP Biology | 5.00 | #1 of 1 | Model card | |
| AP Calculus BC | 4.00 | #1 of 1 | Model card | |
| AP Chemistry | 4.00 | #1 of 1 | Model card | |
| AP English Language and Composition | 2.00 | #1 of 1 | Model card | |
| AP English Literature and Composition | 2.00 | #1 of 1 | Model card | |
| AP Environmental Science | 5.00 | #1 of 1 | Model card | |
| AP Macroeconomics | 5.00 | #1 of 1 | Model card | |
| AP Microeconomics | 5.00 | #1 of 1 | Model card | |
| AP Physics 2 | 4.00 | #1 of 1 | Model card | |
| AP Psychology | 5.00 | #1 of 1 | Model card | |
| AP Statistics | 5.00 | #1 of 1 | Model card | |
| AP US Government | 5.00 | #1 of 1 | Model card | |
| AP US History | 5.00 | #1 of 1 | Model card | |
| AP World History | 4.00 | #1 of 1 | Model card | |
| MMLU calibration (expected calibration error) | 0.07 | #1 of 1 | Model card | |
| Preference on ChatGPT and API prompts | 70.2% | #1 of 1 | Model card | |
| SAT Evidence-Based Reading & Writing | 710.0 | #1 of 1 | Model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GRE Quantitative | 163.0 | #1 of 6 | Model card | |
| ZebraLogic | 27.1% | #14 of 46 | Official leaderboard | |
| GSM8K | 92% | #30 of 87 | Model card | |
| GPQA Diamond | 35.7% | #185 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 0.56% | #197 of 202 | Epoch AI | |
| AMC 10 2022 | 30.0 | #1 of 1 | Model card | |
| AMC 12 2022 | 60.0 | #1 of 1 | Model card | |
| SAT Math | 700.0 | #1 of 1 | Model card | |
| USABO Semifinal Exam 2020 | 87.0 | #1 of 1 | Model card | |
| USNCO Local Section Exam 2022 | 36.0 | #1 of 1 | Model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval+ | 79.3% | #12 of 46 | Official leaderboard | |
| Spider | 85.3% | #2 of 5 | Official leaderboard | |
| HumanEval | 67% | #70 of 95 | Model card | |
| METR Time Horizon 1.1 | 4.0 min | #21 of 26 | Official leaderboard | |
| METR Time Horizon | 6.0 min | #26 of 31 | Official leaderboard | |
| Codeforces rating | 392 | #41 of 41 | Model card | |
| Leetcode (easy) | 75.6% | #1 of 1 | Model card | |
| Leetcode (hard) | 6.70% | #1 of 1 | Model card | |
| Leetcode (medium) | 26.3% | #1 of 1 | Model card |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PubMedQA | 75.2% | #2 of 3 | Official leaderboard | |
| Advanced Sommelier (theory knowledge) | 77% | #1 of 1 | Model card | |
| Certified Sommelier (theory knowledge) | 86% | #1 of 1 | Model card | |
| Introductory Sommelier (theory knowledge) | 92% | #1 of 1 | Model card | |
| Medical Knowledge Self-Assessment Program (MKSAP) | 75% | #1 of 1 | Model card | |
| Uniform Bar Exam (MBE+MEE+MPT) | 298.0 | #1 of 1 | Model card |
Vision, audio and robotics
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DecodingTrust | 69.2 | #4 of 5 | Official leaderboard | |
| RealToxicityPrompts | 0.73% | #2 of 2 | Model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AGI Score (A Definition of AGI) | 27% | #2 of 2 | Official leaderboard |
OpenAI's models over time
OpenAI's models in color, against every other model on the Epoch Capabilities Index, by release date.
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 43 of them aren't in the launch post.
Capabilities
- Uniform Bar Exam (MBE+MEE+MPT) 298
- LSAT 163
- SAT Evidence-Based Reading & Writing 710
- SAT Math 700
- GRE Quantitative 163
- GRE Verbal 169
- GRE Analytical Writing 4
- USABO Semifinal Exam 2020 87
- USNCO Local Section Exam 2022 36
- Medical Knowledge Self-Assessment Program (MKSAP) 75
- Codeforces rating 392
- AP Art History 5
- AP Biology 5
- AP Calculus BC 4
- AP Chemistry 4
- AP English Language and Composition 2
- AP English Literature and Composition 2
- AP Environmental Science 5
- AP Macroeconomics 5
- AP Microeconomics 5
- AP Physics 2 4
- AP Psychology 5
- AP Statistics 5
- AP US Government 5
- AP US History 5
- AP World History 4
- AMC 10 2022 30
- AMC 12 2022 60
- Introductory Sommelier (theory knowledge) 92
- Certified Sommelier (theory knowledge) 86
- Advanced Sommelier (theory knowledge) 77
- Leetcode (easy) 75.6
- Leetcode (medium) 26.3
- Leetcode (hard) 6.7
- MMLU 86.4
- HellaSwag 95.3
- ARC-Challenge 96.3
- WinoGrande 87.5
- HumanEval 67
- DROP 80.9
- GSM8K 92
- MMLU translated with Azure Translate
- Hindsight Neglect
- Preference on ChatGPT and API prompts 70.2