Claude 3.5 Sonnet ranks first on 6 of the 120 benchmarks it has scores on.
Closed weights Mid-size
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| EQ-Bench v2 | 86.4 | #1 of 85 | Official leaderboard | |
| MixEval | 89.9% | #1 of 34 | Official leaderboard | |
| ThaiExam | 75.1% | #1 of 24 | Official leaderboard | |
| BIG-Bench Hard | 93.1% | #3 of 71 | Model card | |
| MMLU | 90.4% | #11 of 127 | Model card | |
| MGSM | 91.6% | #6 of 52 | Model card | |
| WildBench | 54.5 | #6 of 47 | Official leaderboard | |
| FACTS Grounding | 83.3% | #5 of 34 | Official leaderboard | |
| LLM-AggreFact | 77.2% | #1 of 6 | Official leaderboard | |
| Chinese SimpleQA | 53.5% | #5 of 27 | Official leaderboard | |
| BullshitBench | 52.7% | #24 of 107 | Official leaderboard | |
| BuzzBench | 61.9 | #6 of 23 | Official leaderboard | |
| SEAL Precise Instruction Following | 86.0 | #5 of 18 | Official leaderboard | |
| MMBench-CN v1.1 | 79.3% | #53 of 178 | Official leaderboard | |
| DROP | 87.1% | #13 of 40 | Model card | |
| MMLU-Pro | 76.1% | #69 of 212 | Official leaderboard | |
| Needle In A Haystack | 99.7% | #3 of 9 | Model card | |
| Honesty human feedback win rate (Anthropic) | 66% | #2 of 6 | Model card | |
| RewardBench | 84.2% | #19 of 48 | Official leaderboard | |
| BullshitBench V2 | 46% | #51 of 122 | Official leaderboard | |
| ECLeKTic | 20% | #15 of 35 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.60% | #67 of 148 | Official leaderboard | |
| LMArena (Elo) | 1,343 | #182 of 333 | Official leaderboard | |
| Longform Creative Writing (EQ-Bench) | 58.4 | #67 of 119 | Official leaderboard | |
| Creative Writing v3 | 64.2 | #73 of 120 | Official leaderboard | |
| RewardBench 2 | 64.7% | #27 of 42 | Official leaderboard | |
| EQ-Bench 3 | 1,102 | #52 of 78 | Official leaderboard | |
| SEAL Multilingual | 992 | #13 of 19 | Official leaderboard | |
| MultiLoKo | 23.2% | #5 of 7 | Official leaderboard | |
| WritingBench | 53.8 | #36 of 45 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 96.4% | #4 of 87 | Model card | |
| GSM1k | 96.6% | #1 of 15 | Official leaderboard | |
| OlympicArena | 39.2% | #2 of 25 | Official leaderboard | |
| KOR-Bench | 59% | #3 of 33 | Official leaderboard | |
| ForecastBench | 55.6 | #14 of 86 | Official leaderboard | |
| ZebraLogic | 33.4% | #8 of 46 | Official leaderboard | |
| MATH | 71.1% | #39 of 93 | Model card | |
| MATH Level 5 | 51.7% | #49 of 94 | Epoch AI | |
| Omni-MATH | 26.2% | #5 of 9 | Official leaderboard | |
| GPQA Diamond | 54% | #139 of 219 | Epoch AI | |
| DTBench | 46.3% | #118 of 178 | Official leaderboard | |
| SimpleBench | 27.5% | #80 of 100 | Official leaderboard | |
| OTIS Mock AIME 2024-2025 | 6.53% | #165 of 202 | Epoch AI | |
| FrontierMath Tier 4 | 0% | #46 of 55 | Epoch AI | |
| FrontierMath | 1.03% | #59 of 70 | Epoch AI | |
| AIME 2025 | 3.33% | #53 of 53 | Official leaderboard | |
| HMMT 2025 | 1.67% | #53 of 53 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Aider-Edit | 84.2% | #1 of 37 | Official leaderboard | |
| HumanEval | 92% | #9 of 95 | Model card | |
| BigCodeBench | 52.7% | #9 of 69 | Official leaderboard | |
| SWE-bench Lite | 51.3% | #2 of 15 | Official leaderboard | |
| SciCode | 4.60% | #3 of 18 | Official leaderboard | |
| MBPP+ | 74.3% | #6 of 35 | Official leaderboard | |
| BigCodeBench-Hard | 30.4% | #19 of 96 | Official leaderboard | |
| MHPP | 45.2% | #5 of 23 | Official leaderboard | |
| HumanEval+ | 81.7% | #11 of 46 | Official leaderboard | |
| SWE-bench | 24.9% | #3 of 9 | Official leaderboard | |
| SEAL Coding | 1,079 | #9 of 26 | Official leaderboard | |
| SWE-bench Verified | 63.4% | #26 of 60 | Official leaderboard | |
| Aider Polyglot | 51.6% | #21 of 46 | Official leaderboard | |
| METR Time Horizon | 20 min | #20 of 31 | Official leaderboard | |
| SWT-bench | 12.3% | #6 of 9 | Official leaderboard | |
| WeirdML | 40% | #77 of 112 | Official leaderboard | |
| METR Time Horizon 1.1 | 11 min | #18 of 26 | Official leaderboard | |
| BIRD-CRITIC | 21.5% | #7 of 9 | Official leaderboard | |
| SWE-bench Multimodal | 12.2% | #8 of 9 | Official leaderboard | |
| MLE-bench | 7.56% | #11 of 12 | Official leaderboard | |
| Internal agentic coding evaluation (Anthropic) | 64% | #3 of 3 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GAIA | 65.1% | #22 of 57 | Official leaderboard | |
| BALROG | 32.6% | #14 of 35 | Official leaderboard | |
| CRMArena | 41.8% | #4 of 9 | Official leaderboard | |
| ToolComp | 59.4% | #17 of 32 | Official leaderboard | |
| AceBench | 68.9% | #9 of 16 | Official leaderboard | |
| LisanBench | 365.3 | #75 of 133 | Official leaderboard | |
| LLM Chess | -134 | #84 of 124 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MortgageTax | 64.1% | #45 of 86 | Official leaderboard | |
| MedXpertQA-MM | 33.2% | #6 of 9 | Official leaderboard | |
| TaxEval v2 | 70.2% | #87 of 129 | Official leaderboard | |
| FailureSensorIQ | 58.6% | #17 of 24 | Official leaderboard | |
| CorpFin v2 | 53.6% | #90 of 119 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| CharXiv | 60.2% | #4 of 77 | Official leaderboard | |
| ChartQA | 90.8% | #2 of 37 | Model card | |
| AI2D | 94.7% | #4 of 44 | Model card | |
| MEGA-Bench | 52.1% | #3 of 22 | Official leaderboard | |
| Image2Struct | 73.7 | #4 of 18 | Official leaderboard | |
| VCR (Visual Caption Restoration) | 35.6% | #10 of 42 | Official leaderboard | |
| MMStar | 62.2% | #46 of 173 | Official leaderboard | |
| DocVQA | 95.2% | #9 of 33 | Model card | |
| MMMU | 68.3% | #38 of 136 | Official leaderboard | |
| OCRBench v2 | 47.5% | #22 of 65 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 54.1% | #53 of 152 | Official leaderboard | |
| VisualPuzzles | 42.4% | #11 of 29 | Official leaderboard | |
| MMBench | 79.7% | #59 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 77.7% | #73 of 178 | Official leaderboard | |
| MMMU-Pro | 51.5% | #25 of 57 | Official leaderboard | |
| TOMATO | 27.8% | #11 of 22 | Official leaderboard | |
| MATH-Vision (MathVision) | 38% | #63 of 105 | Official leaderboard | |
| VHELM | 48.2 | #25 of 41 | Official leaderboard | |
| Video-MME | 62.9% | #17 of 26 | Official leaderboard | |
| VISTA | 38.4% | #36 of 54 | Official leaderboard | |
| GeoBench | 62% | #33 of 48 | Official leaderboard | |
| MathVista | 67.7% | #47 of 64 | Model card | |
| Vision Arena | 1,145 | #102 of 136 | Official leaderboard | |
| ZeroBench | 2.00% | #53 of 66 | Official leaderboard | |
| VPCT | 33% | #26 of 28 | Epoch AI |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SimpleSafetyTests | 100% | #4 of 75 | Official leaderboard | |
| HELM Safety | 97.7 | #8 of 75 | Official leaderboard | |
| Situational Awareness Dataset (SAD) | 57.1% | #2 of 17 | Official leaderboard | |
| Anthropic Red Team | 99.8% | #9 of 75 | Official leaderboard | |
| FORTRESS | 13% | #8 of 66 | Official leaderboard | |
| XSTest | 1.70% | #2 of 16 | Model card | |
| AIR-Bench 2024 | 85.9% | #11 of 82 | Official leaderboard | |
| WildChat toxic refusals (Anthropic) | 96.4% | #1 of 3 | Model card | |
| BaxBench | 36% | #12 of 34 | Official leaderboard | |
| Spiral-Bench v1.2 | 42.6 | #9 of 21 | Official leaderboard | |
| OR-Bench | 43.8% | #13 of 28 | Official leaderboard | |
| Harmlessness human feedback win rate (Anthropic) | 52% | #2 of 4 | Model card | |
| Cybench | 17.5% | #15 of 24 | Official leaderboard | |
| SEAL Adversarial Robustness | 16.0 | #5 of 8 | Official leaderboard | |
| Spiral-Bench v1.0 | 47.7 | #10 of 13 | Official leaderboard | |
| AgentDojo | 33.9% | #19 of 20 | Official leaderboard | |
| WildChat non-toxic incorrect refusals (Anthropic) | 11% | #3 of 3 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 88.5 | #4 of 71 | Official leaderboard | |
| OpenVLM Leaderboard | 67.9 | #45 of 173 | Official leaderboard |
Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.
The benchmarks the lab chose to headline, with the scores it gave.
Every result in the longer documents the lab published with it. 8 of them aren't in the launch post.