Claude 3 Sonnet ranks first on Honesty human feedback win rate (Anthropic).
Closed weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) | 16.3% | #7 of 148 | Official leaderboard | |
| Honesty human feedback win rate (Anthropic) | 69% | #1 of 6 | Model card | |
| HellaSwag | 89% | #8 of 47 | Model card | |
| ARC-Challenge | 93.2% | #9 of 45 | Model card | |
| RepoQA | 87.4% | #5 of 22 | Official leaderboard | |
| EQ-Bench v2 | 80.5 | #23 of 85 | Official leaderboard | |
| RACE-H | 88.8% | #2 of 7 | Model card | |
| AlpacaEval 2.0 | 34.9% | #12 of 40 | Official leaderboard | |
| ThaiExam | 57.9% | #9 of 24 | Official leaderboard | |
| BIG-Bench Hard | 82.9% | #28 of 71 | Model card | |
| WildBench | 45.2 | #20 of 47 | Official leaderboard | |
| QuALITY | 85.9% | #3 of 7 | Model card | |
| MixEval | 81.7% | #15 of 34 | Official leaderboard | |
| MGSM | 83.5% | #27 of 52 | Model card | |
| MMLU | 81.5% | #66 of 127 | Model card | |
| MMLU-Pro | 56.8% | #119 of 212 | Official leaderboard | |
| WinoGrande | 75.1% | #26 of 42 | Model card | |
| RewardBench | 74.6% | #34 of 48 | Official leaderboard | |
| LMArena (Elo) | 1,281 | #244 of 333 | Official leaderboard | |
| MMBench-CN v1.1 | 63.5% | #132 of 178 | Official leaderboard | |
| Okapi MMLU | 69% | #6 of 8 | Model card | |
| LSAT | 158.3 | #3 of 4 | Model card | |
| Needle In A Haystack | 95.4% | #9 of 9 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 92.3% | #28 of 87 | Model card | |
| OlympicArena | 25.5% | #10 of 25 | Official leaderboard | |
| ZebraLogic | 18.7% | #22 of 46 | Official leaderboard | |
| GSM1k | 93.3% | #8 of 15 | Official leaderboard | |
| AMC 8 2023 | 54.0 | #2 of 3 | Model card | |
| GPQA Extended | 38% | #2 of 3 | Model card | |
| GPQA Main | 38.5% | #5 of 7 | Model card | |
| GPQA Diamond | 40.6% | #174 of 219 | Epoch AI | |
| MATH | 40.5% | #75 of 93 | Model card | |
| MATH Level 5 | 18.2% | #78 of 94 | Epoch AI | |
| DTBench | 22.7% | #153 of 178 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 2.50% | #182 of 202 | Epoch AI | |
| AMC 10 2023 | 24.0 | #3 of 3 | Model card | |
| AMC 12 2023 | 27.0 | #3 of 3 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP | 79.4% | #22 of 66 | Model card | |
| MBPP+ | 69.3% | #14 of 35 | Official leaderboard | |
| BigCodeBench | 48.3% | #28 of 69 | Official leaderboard | |
| HumanEval+ | 64% | #23 of 46 | Official leaderboard | |
| SciCode | 1.50% | #9 of 18 | Official leaderboard | |
| BigCodeBench-Hard | 23.4% | #52 of 96 | Official leaderboard | |
| HumanEval | 73% | #62 of 95 | Model card | |
| APPS | 55.9% | #2 of 3 | Model card | |
| Aider-Edit | 54.9% | #28 of 37 | Official leaderboard | |
| SEAL Coding | 879 | #22 of 26 | Official leaderboard | |
| WeirdML | 10.2% | #110 of 112 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PubMedQA | 78.3% | #1 of 3 | Model card | |
| MBE (Multistate Bar Examination) | 71% | #3 of 4 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AI2D | 88.7% | #21 of 44 | Model card | |
| Image2Struct | 49.1 | #9 of 18 | Official leaderboard | |
| CharXiv | 32.2% | #39 of 77 | Official leaderboard | |
| MMMU | 53.1% | #76 of 136 | Official leaderboard | |
| MTVQA-Test | 21.1% | #11 of 18 | Official leaderboard | |
| MMStar | 44.2% | #129 of 173 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 27.8% | #119 of 152 | Official leaderboard | |
| MMBench | 67.8% | #122 of 155 | Official leaderboard | |
| VHELM | 29.8 | #33 of 41 | Official leaderboard | |
| ChartQA | 81.1% | #30 of 37 | Model card | |
| DocVQA | 89.5% | #27 of 33 | Model card | |
| MMBench-EN v1.1 | 64.2% | #149 of 178 | Official leaderboard | |
| ZeroBench | 1.00% | #58 of 66 | Official leaderboard | |
| Vision Arena | 1,016 | #127 of 136 | Official leaderboard | |
| MathVista | 47.9% | #62 of 64 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SimpleSafetyTests | 100% | #2 of 75 | Official leaderboard | |
| Anthropic Red Team | 99.8% | #7 of 75 | Official leaderboard | |
| AIR-Bench 2024 | 84.7% | #14 of 82 | Official leaderboard | |
| Harmlessness human feedback win rate (Anthropic) | 87% | #1 of 4 | Model card | |
| HELM Safety | 94.3 | #27 of 75 | Official leaderboard | |
| Situational Awareness Dataset (SAD) | 50% | #7 of 17 | Official leaderboard | |
| Multimodal policy red-teaming (Anthropic) | 99.2% | #1 of 2 | Model card | |
| AgentDojo | 26.7% | #15 of 20 | Official leaderboard | |
| OR-Bench | 94.5% | #25 of 28 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 53.5 | #114 of 173 | Official leaderboard | |
| HELM Lite | 37.6 | #52 of 71 | Official leaderboard |
Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.
Every result in the longer documents the lab published with it. 41 of them aren't in the launch post.