Claude 3 Haiku ranks first on 2 of the 74 benchmarks it has scores on.
Closed weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| RepoQA | 81.8% | #8 of 22 | Official leaderboard | |
| ARC-Challenge | 89.2% | #17 of 45 | Model card | |
| HellaSwag | 85.9% | #20 of 47 | Model card | |
| ThaiExam | 54% | #13 of 24 | Official leaderboard | |
| BIG-Bench Hard | 73.7% | #40 of 71 | Model card | |
| RACE-H | 87% | #4 of 7 | Model card | |
| WildBench | 40.3 | #27 of 47 | Official leaderboard | |
| MixEval | 79.7% | #22 of 34 | Official leaderboard | |
| MGSM | 75.1% | #34 of 52 | Model card | |
| WinoGrande | 74.2% | #28 of 42 | Model card | |
| MMLU | 76.7% | #86 of 127 | Model card | |
| BullshitBench | 18.2% | #73 of 107 | Official leaderboard | |
| EQ-Bench v2 | 63.6 | #60 of 85 | Official leaderboard | |
| MMLU-Pro | 42.3% | #155 of 212 | Official leaderboard | |
| LMArena (Elo) | 1,262 | #256 of 333 | Official leaderboard | |
| RewardBench | 72.9% | #38 of 48 | Official leaderboard | |
| QuALITY | 80.2% | #6 of 7 | Model card | |
| BullshitBench V2 | 10% | #106 of 122 | Official leaderboard | |
| MMBench-CN v1.1 | 56.2% | #155 of 178 | Official leaderboard | |
| Needle In A Haystack | 95.9% | #8 of 9 | Model card | |
| Creative Writing v3 | 48.3 | #109 of 120 | Official leaderboard | |
| LLM Confabulation Leaderboard | 34.2% | #55 of 58 | Official leaderboard | |
| RewardBench 2 | 37.1% | #42 of 42 | Official leaderboard | |
| Okapi MMLU | 65.2% | #8 of 8 | Model card | |
| LSAT | 156.3 | #4 of 4 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ForecastBench | 44.7 | #1 of 86 | Official leaderboard | |
| GSM8K | 88.9% | #42 of 87 | Model card | |
| ZebraLogic | 14.3% | #27 of 46 | Official leaderboard | |
| AMC 10 2023 | 54.0 | #2 of 3 | Model card | |
| AMC 12 2023 | 48.0 | #2 of 3 | Model card | |
| MATH | 40.9% | #74 of 93 | Model card | |
| GPQA Diamond | 36.3% | #184 of 219 | Epoch AI | |
| GPQA Main | 35.2% | #6 of 7 | Model card | |
| MATH Level 5 | 14.9% | #82 of 94 | Epoch AI | |
| DTBench | 16.9% | #162 of 178 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 1.81% | #189 of 202 | Epoch AI | |
| AMC 8 2023 | 36.0 | #3 of 3 | Model card | |
| GPQA Extended | 34.7% | #3 of 3 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP | 80.4% | #16 of 66 | Model card | |
| HumanEval+ | 68.9% | #20 of 46 | Official leaderboard | |
| MBPP+ | 68.8% | #16 of 35 | Official leaderboard | |
| BigCodeBench | 44.8% | #36 of 69 | Official leaderboard | |
| HumanEval | 75.9% | #55 of 95 | Model card | |
| MHPP | 24.8% | #14 of 23 | Official leaderboard | |
| BigCodeBench-Hard | 18.3% | #64 of 96 | Official leaderboard | |
| SWT-bench | 2.50% | #8 of 9 | Official leaderboard | |
| Aider-Edit | 47.4% | #33 of 37 | Official leaderboard | |
| WeirdML | 9.84% | #111 of 112 | Official leaderboard | |
| LiveCodeBench v5 | 18.5% | #21 of 21 | Official leaderboard | |
| LiveCodeBench v6 | 20.2% | #21 of 21 | Official leaderboard | |
| APPS | 54.8% | #3 of 3 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LisanBench | 135.7 | #104 of 133 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PubMedQA | 76% | #2 of 3 | Model card | |
| CadEval | 12% | #15 of 15 | Epoch AI | |
| MBE (Multistate Bar Examination) | 64% | #4 of 4 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ScienceQA | 72% | #11 of 23 | Epoch AI | |
| CharXiv | 31.8% | #40 of 77 | Official leaderboard | |
| TOMATO | 26.2% | #12 of 22 | Official leaderboard | |
| MMMU | 50.2% | #84 of 136 | Official leaderboard | |
| ChartQA | 81.7% | #26 of 37 | Model card | |
| AI2D | 86.7% | #31 of 44 | Model card | |
| CCBench (MMBench Chinese Culture) | 24.5% | #126 of 152 | Official leaderboard | |
| VHELM | 27.1 | #36 of 41 | Official leaderboard | |
| DocVQA | 88.8% | #29 of 33 | Model card | |
| MMStar | 38.1% | #154 of 173 | Official leaderboard | |
| MMBench | 60.7% | #142 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 58% | #164 of 178 | Official leaderboard | |
| ZeroBench | 0% | #62 of 66 | Official leaderboard | |
| Vision Arena | 1,000 | #129 of 136 | Official leaderboard | |
| MathVista | 46.4% | #63 of 64 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Anthropic Red Team | 100% | #1 of 75 | Official leaderboard | |
| SimpleSafetyTests | 100% | #3 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 82.7% | #18 of 82 | Official leaderboard | |
| AgentDojo | 9.06% | #7 of 20 | Official leaderboard | |
| Situational Awareness Dataset (SAD) | 45.9% | #11 of 17 | Official leaderboard | |
| HELM Safety | 87.8 | #56 of 75 | Official leaderboard | |
| OR-Bench | 96.3% | #27 of 28 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 50.6 | #123 of 173 | Official leaderboard | |
| HELM Lite | 26.3 | #64 of 71 | Official leaderboard | |
| LMCA | 8.84% | #129 of 139 | Epoch AI | |
| Conceptual Reasoning Index | 11.6 | #144 of 149 | Official leaderboard | |
| ACCoRD | 14.7 | #147 of 150 | Official leaderboard |
Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.
Every result in the longer documents the lab published with it. 37 of them aren't in the launch post.