Claude 3.5 Haiku places in the top three on 2 benchmarks.
Closed weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BullshitBench | 58.2% | #20 of 107 | Official leaderboard | |
| BIG-Bench Hard | 86.6% | #16 of 71 | Model card | |
| BullshitBench V2 | 50% | #42 of 122 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.90% | #57 of 148 | Official leaderboard | |
| Phare | 69.1% | #24 of 62 | Official leaderboard | |
| BuzzBench | 53.3 | #9 of 23 | Official leaderboard | |
| MGSM | 85.6% | #24 of 52 | Model card | |
| FACTS Grounding | 78.1% | #16 of 34 | Official leaderboard | |
| IFEval | 85.9% | #68 of 126 | Model card | |
| MMLU | 80.9% | #69 of 127 | Model card | |
| LMArena (Elo) | 1,325 | #200 of 333 | Official leaderboard | |
| Lech Mazur Creative Writing | 7.35 | #31 of 44 | Epoch AI | |
| ToRR | 42.7 | #10 of 14 | Official leaderboard | |
| ECLeKTic | 10.1% | #26 of 35 | Official leaderboard | |
| MMLU-Pro | 65% | #122 of 151 | Model card | |
| Creative Writing v3 | 54.1 | #99 of 120 | Official leaderboard | |
| WritingBench | 49.1 | #40 of 45 | Official leaderboard | |
| LLM Confabulation Leaderboard | 36.7% | #56 of 58 | Official leaderboard | |
| Global MMLU Lite | 61.1% | #42 of 43 | Official leaderboard | |
| Honesty human feedback win rate (Anthropic) | 42% | #6 of 6 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH | 69.2% | #44 of 93 | Model card | |
| ZebraLogic | 18.7% | #23 of 46 | Official leaderboard | |
| MATH Level 5 | 46.4% | #56 of 94 | Epoch AI | |
| CritPt | 0% | #117 of 160 | Epoch AI | |
| GPQA Diamond | 38.1% | #179 of 219 | Epoch AI | |
| DTBench | 27.8% | #147 of 178 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 4.31% | #176 of 202 | Epoch AI | |
| FrontierMath | 0.34% | #66 of 70 | Epoch AI | |
| AIME 2024 | 5.30% | #87 of 87 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Aider-Edit | 75.2% | #4 of 37 | Official leaderboard | |
| BigCodeBench | 52.5% | #10 of 69 | Official leaderboard | |
| BigCodeBench-Hard | 30.1% | #20 of 96 | Official leaderboard | |
| HumanEval | 88.1% | #25 of 95 | Model card | |
| SWE-PolyBench | 9.86% | #3 of 6 | Official leaderboard | |
| Internal agentic coding evaluation (Anthropic) | 74% | #2 of 3 | Model card | |
| SWE-bench Verified | 40.6% | #41 of 60 | Official leaderboard | |
| Aider Polyglot | 28% | #38 of 54 | Epoch AI | |
| WeirdML | 30.7% | #98 of 112 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | 53.7% | #39 of 67 | Official leaderboard | |
| LLM Chess | -120 | #79 of 124 | Official leaderboard | |
| BALROG | 19.3% | #24 of 35 | Official leaderboard | |
| LisanBench | 144.0 | #101 of 133 | Official leaderboard | |
| Elimination Game | 3 | #45 of 56 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.77 | #26 of 31 | Official leaderboard | |
| Step Race | 1 | #58 of 59 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CadEval | 32% | #10 of 15 | Epoch AI | |
| FailureSensorIQ | 57.6% | #18 of 24 | Official leaderboard | |
| Vals Finance Agent v1 | 14.3% | #36 of 47 | Official leaderboard | |
| CorpFin v2 | 50.8% | #95 of 119 | Official leaderboard | |
| LegalBench | 70.3% | #114 of 138 | Official leaderboard | |
| TaxEval v2 | 57.4% | #121 of 129 | Official leaderboard | |
| ProfBench | 21.2% | #72 of 72 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vision Arena | 1,127 | #108 of 136 | Official leaderboard | |
| GeoBench | 34% | #48 of 48 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| XSTest | 2.00% | #3 of 16 | Model card | |
| WildChat non-toxic incorrect refusals (Anthropic) | 4.80% | #1 of 3 | Model card | |
| KORA | 54% | #15 of 42 | Official leaderboard | |
| FORTRESS | 30.4% | #42 of 66 | Official leaderboard | |
| Harmlessness human feedback win rate (Anthropic) | 43% | #4 of 4 | Model card | |
| WildChat toxic refusals (Anthropic) | 88% | #3 of 3 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 53.1 | #34 of 71 | Official leaderboard | |
| LiveBench | 43.5% | #35 of 48 | Epoch AI | |
| HELM Capabilities | 54.9 | #51 of 60 | Official leaderboard |
Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.
Every result in the longer documents the lab published with it. 19 of them aren't in the launch post.