Claude 3.7 Sonnet

Claude 3.7 Sonnet ranks first on 2 of the 131 benchmarks it has scores on.

Anthropic Released 24 Feb 2025 Family Claude 3.7 The lab's model card

137

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Anthropic Feb 2025

Claude 3.7 Sonnet

Closed weights Mid-size

141.0 #119 of 271

Epoch Capabilities Index

Humanity's Last Exam 8.04% #34 /47
SWE-bench Verified 61% #28 /32
MultiChallenge 51.6% #27 /35

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

Benchmark	Score	Rank	Source
IFEval	93.2%	#13 of 126	Model card
BuzzBench	68.2	#3 of 23	Official leaderboard
Fiction.LiveBench	83.3%	#9 of 57	Epoch AI
MMLU-Pro	84%	#36 of 212	Official leaderboard
BullshitBench	52.7%	#25 of 107	Official leaderboard
RewardBench 2	75.4%	#10 of 42	Official leaderboard
MMBench-CN v1.1	79.5%	#51 of 178	Official leaderboard
Arena-Hard v2.0	59.8%	#7 of 23	Official leaderboard
ECLeKTic	23.3%	#11 of 35	Official leaderboard
Lech Mazur Creative Writing	8.11	#15 of 44	Epoch AI
LLM Confabulation Leaderboard	14.7%	#21 of 58	Official leaderboard
BullshitBench V2	49%	#45 of 122	Official leaderboard
Phare	69.5%	#23 of 62	Official leaderboard
FaithJudge	16.1%	#14 of 37	Official leaderboard
FACTS Grounding	78.8%	#14 of 34	Official leaderboard
MMMLU	86.1%	#26 of 62	Model card
LMArena (Elo)	1,388	#143 of 333	Official leaderboard
WritingBench	73.4	#20 of 45	Official leaderboard
Vectara Hallucination Leaderboard (HHEM)	4.50%	#71 of 148	Official leaderboard
MultiNRC	27.8%	#20 of 38	Official leaderboard
Longform Creative Writing (EQ-Bench)	57.7	#71 of 119	Official leaderboard
Creative Writing v3	64.7	#72 of 120	Official leaderboard
EQ-Bench 3	1,128	#48 of 78	Official leaderboard
Global MMLU Lite	80.8%	#31 of 43	Official leaderboard
Humanity's Last Exam	8.04%	#34 of 47	Official leaderboard
MultiChallenge	51.6%	#27 of 35	Official leaderboard

Math and science

Benchmark	Score	Rank	Source
MATH Level 5	91.2%	#15 of 94	Epoch AI
Misguided Attention	56.1%	#10 of 45	Official leaderboard
CORE-Bench	37.8%	#6 of 17	Official leaderboard
GPQA Diamond	79.7%	#88 of 219	Epoch AI
PolyMath	33.5%	#18 of 38	Official leaderboard
MATH-500	96.2%	#32 of 67	Model card
AIME 2024	80%	#42 of 87	Model card
OTIS Mock AIME 2024-2025	57.8%	#118 of 202	Epoch AI
SimpleBench	46.4%	#59 of 100	Official leaderboard
ForecastBench	63.8	#57 of 86	Official leaderboard
FrontierMath	4.14%	#49 of 70	Epoch AI
USAMO 2025	3.65%	#7 of 10	Official leaderboard
ARC-AGI	28.6%	#74 of 92	Official leaderboard
PHYBench	15.3%	#23 of 28	Official leaderboard
ARC-AGI-2	0.90%	#81 of 95	Official leaderboard
PutnamBench	0%	#12 of 14	Official leaderboard
HMMT 2025	31.7%	#46 of 53	Official leaderboard
AIME 2025	49.2%	#47 of 53	Official leaderboard
SMT 2025	56.6%	#39 of 40	Official leaderboard
BRUMO 2025	65.8%	#42 of 42	Official leaderboard

Coding

Benchmark	Score	Rank	Source
BigCodeBench-Hard	35.8%	#1 of 96	Official leaderboard
Multi-SWE-bench	9.25%	#2 of 11	Official leaderboard
SWE-bench	33.8%	#2 of 9	Official leaderboard
SWT-bench	52.5%	#2 of 9	Official leaderboard
Aider Polyglot	64.9%	#13 of 54	Epoch AI
SWE-bench Lite	48%	#4 of 15	Official leaderboard
METR Time Horizon	55 min	#11 of 31	Official leaderboard
SWE-bench Multimodal	31.3%	#4 of 9	Official leaderboard
ArtifactsBench	51.3%	#9 of 20	Official leaderboard
METR Time Horizon 1.1	60 min	#14 of 26	Official leaderboard
LiveSQLBench	21.7%	#24 of 36	Official leaderboard
SWE-bench-Live	17.7%	#10 of 13	Official leaderboard
GSO-Bench	3.80%	#26 of 30	Official leaderboard
SWE-bench Verified	61%	#28 of 32	Epoch AI
SWE-fficiency	2.44%	#17 of 17	Official leaderboard
SWE-bench Verified (hard subset)	23%	#9 of 9	Model card
METR data deduplication	13.3%	#3 of 3	Model card
RE-Bench subset (Anthropic)	0.24	#1 of 1	Model card

Agents and tools

Benchmark	Score	Rank	Source
Galileo Agent Leaderboard	0.95	#1 of 31	Official leaderboard
Elimination Game	5	#9 of 56	Official leaderboard
tau-bench	69.8%	#4 of 23	Model card
GAIA	74.8%	#12 of 57	Official leaderboard
ToolComp	65.3%	#7 of 32	Official leaderboard
AgentBench FC	53.2%	#3 of 12	Official leaderboard
SnakeBench	27	#53 of 183	Official leaderboard
Berkeley Function Calling Leaderboard (BFCL)	58.6%	#27 of 67	Official leaderboard
Step Race	3	#25 of 59	Official leaderboard
LisanBench	498.3	#67 of 133	Official leaderboard
LLM Chess	-1	#64 of 124	Official leaderboard
MCP-Universe	24.2%	#18 of 30	Official leaderboard
OSWorld	35.8%	#6 of 9	Epoch AI
OSWorld-Verified	35.8%	#23 of 33	Official leaderboard
tau2-bench	61.8%	#7 of 10	Official leaderboard
DeepResearch Bench	43.6%	#17 of 24	Epoch AI
VideoGameBench	0%	#4 of 5	Official leaderboard

Professional work

Benchmark	Score	Rank	Source
LEXam	62.9%	#3 of 31	Official leaderboard
MortgageTax	68.7%	#12 of 86	Official leaderboard
xbench-Recruitment	61.4	#2 of 8	Official leaderboard
Vals Finance Agent v1	44.1%	#12 of 47	Official leaderboard
TheAgentCompany	30.9%	#4 of 14	Epoch AI
xbench-Marketing	47.6	#2 of 7	Official leaderboard
TaxEval v2	74%	#41 of 129	Official leaderboard
CadEval	54%	#6 of 15	Epoch AI
LegalBench	82.5%	#68 of 138	Official leaderboard
MedHELM	52.9	#7 of 13	Official leaderboard
CorpFin v2	60.4%	#68 of 119	Official leaderboard

Vision, audio and robotics

Benchmark	Score	Rank	Source
CharXiv	64.2%	#3 of 77	Official leaderboard
MMMU	75%	#21 of 136	Official leaderboard
MMStar	65.1%	#33 of 173	Official leaderboard
VisualPuzzles	48.3%	#6 of 29	Official leaderboard
MMBench	81.4%	#44 of 155	Official leaderboard
VISTA	48.2%	#16 of 54	Official leaderboard
MMBench-EN v1.1	79.8%	#55 of 178	Official leaderboard
CCBench (MMBench Chinese Culture)	55.1%	#51 of 152	Official leaderboard
MATH-Vision (MathVision)	58.6%	#39 of 105	Official leaderboard
ZeroBench	6.00%	#34 of 66	Official leaderboard
A-Bench	70.7%	#66 of 116	Official leaderboard
EnigmaEval	4.23%	#24 of 41	Epoch AI
Vision Arena	1,193	#84 of 136	Official leaderboard
GeoBench	68%	#30 of 48	Official leaderboard
SpatialViz-Bench	33.9%	#5 of 8	Epoch AI
VPCT	39%	#18 of 28	Epoch AI
Point-Bench	22.2%	#14 of 17	Official leaderboard
ERQA	14.3%	#17 of 17	Official leaderboard

Safety and security

Benchmark	Score	Rank	Source
SimpleSafetyTests	100%	#8 of 75	Official leaderboard
Anthropic Red Team	99.7%	#11 of 75	Official leaderboard
BaxBench	46.4%	#5 of 34	Official leaderboard
AIR-Bench 2024	81.8%	#21 of 82	Official leaderboard
AgentDojo	7.31%	#6 of 20	Official leaderboard
Bioweapons knowledge questions	51.5%	#1 of 3	Model card
HELM Safety	94.5	#26 of 75	Official leaderboard
Computer use prompt injection evaluation (Anthropic)	74%	#2 of 5	Model card
BBQ (Bias Benchmark for QA)	84%	#14 of 24	Model card
Cybench	20%	#14 of 24	Official leaderboard
FORTRESS	38%	#43 of 66	Official leaderboard
Crypto CTF challenges (Anthropic)	75%	#4 of 6	Model card
Web CTF challenges (Anthropic)	87.5%	#4 of 6	Model card
Bioweapons acquisition uplift trial (Anthropic)	2.10	#2 of 3	Model card
Virology Capabilities Test (VCT)	30.8%	#5 of 7	Official leaderboard
CyberGym	14.5%	#26 of 33	Official leaderboard
Pwn CTF challenges (Anthropic)	38.5%	#5 of 6	Model card
Network CTF challenges (Anthropic)	0%	#6 of 6	Model card
Rev CTF challenges (Anthropic)	0%	#6 of 6	Model card
Cyber-harness network challenges (Incalmo cyber ranges)	66.7%	#1 of 1	Model card
Long-form virology tasks	69.7%	#1 of 1	Model card

Other

Benchmark	Score	Rank	Source
LiveBench	76.1%	#3 of 48	Epoch AI
OpenVLM Leaderboard	70.1	#36 of 173	Official leaderboard
CAIS Risk Index	46.0	#23 of 65	Official leaderboard
HELM Capabilities	67.4	#26 of 60	Official leaderboard
CAIS Text Capabilities Index	8.00	#64 of 66	Official leaderboard
CAIS Vision Capabilities Index	33.9	#52 of 53	Official leaderboard

Anthropic's models over time

Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Claude 3.7 Sonnet and Claude Code (launch post) 24 Feb 2025

Its system card

Every result in the longer documents the lab published with it. 28 of them aren't in the launch post.

Claude 3.7 Sonnet

Claude 3.7 Sonnet

Where it ranks

Knowledge and language

Math and science

Coding

Agents and tools

Professional work

Vision, audio and robotics

Safety and security

Other

Anthropic's models over time

Its launch post

Its system card

Safety

Robustness

Bias

Alignment

Monitorability

Bio and chem risk

Cyber risk

AI R&D risk