Benchmark Almanac

Claude Opus 4

Claude Opus 4 ranks first on 3 of the 104 benchmarks it has scores on.

Anthropic Released 22 May 2025 Family Claude 4 The lab's model card
108
Benchmarks with a score
3
Ranked first, out of five or more models
7
In the top three
Anthropic May 2025

Claude Opus 4

Closed weights Flagship

142.4 #109 of 271
Epoch Capabilities Index
  • Humanity's Last Exam 10.7% #29 /47
  • SWE-bench Verified 70.7% #24 /32
  • MultiChallenge 58.6% #19 /35

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
MMLU-Pro 87.3% #12 of 212 Official leaderboard
SuperGPQA 61.4% #8 of 58 Official leaderboard
RewardBench 2 76.5% #6 of 42 Official leaderboard
Lech Mazur Creative Writing 8.36 #7 of 44 Epoch AI
FaithJudge 14.1% #9 of 37 Official leaderboard
MMMLU 88.8% #17 of 62 Model card
ECLeKTic 24% #10 of 35 Official leaderboard
LMArena (Elo) 1,426 #96 of 333 Official leaderboard
WritingBench 75.1 #16 of 45 Official leaderboard
EQ-Bench 3 1,440 #30 of 78 Official leaderboard
Vectara Hallucination Leaderboard (HHEM) 4.80% #59 of 148 Official leaderboard
BullshitBench 32.7% #45 of 107 Official leaderboard
Creative Writing v3 70.0 #55 of 120 Official leaderboard
LLM Confabulation Leaderboard 15.9% #27 of 58 Official leaderboard
MultiNRC 33.9% #19 of 38 Official leaderboard
FACTS Grounding 77.7% #17 of 34 Official leaderboard
BullshitBench V2 34% #66 of 122 Official leaderboard
MultiChallenge 58.6% #19 of 35 Official leaderboard
Fiction.LiveBench 61.1% #33 of 57 Epoch AI
Humanity's Last Exam 10.7% #29 of 47 Official leaderboard
Vectara Hallucination Leaderboard (2025 dataset) 12% #78 of 104 Official leaderboard

Math and science

BenchmarkScoreRankSource
Misguided Attention 64.1% #4 of 45 Official leaderboard
MATH Level 5 85% #22 of 94 Epoch AI
AIME 2025 90% #49 of 143 Model card
SimpleBench 58.8% #40 of 100 Official leaderboard
GPQA Diamond 76.3% #98 of 219 Epoch AI
DTBench 69.3% #82 of 178 Official leaderboard
FrontierMath Tier 4 4.17% #28 of 55 Epoch AI
OTIS Mock AIME 2024-2025 64.4% #111 of 202 Epoch AI
ARC-AGI-2 8.61% #58 of 95 Official leaderboard
CritPt 0.30% #105 of 160 Epoch AI
FrontierMath 4.48% #48 of 70 Epoch AI
HMMT 2025 60% #37 of 53 Official leaderboard
ForecastBench 64.1 #62 of 86 Official leaderboard
ARC-AGI 35.7% #71 of 92 Official leaderboard
AMO-Bench 10.6% #17 of 21 Official leaderboard
BRUMO 2025 81.7% #35 of 42 Official leaderboard

Coding

BenchmarkScoreRankSource
AutoCodeBench 52.4% #1 of 34 Official leaderboard
Aider Polyglot 72% #11 of 54 Epoch AI
METR Time Horizon 84 min #8 of 31 Official leaderboard
METR data deduplication 32.6% #1 of 3 Model card
METR Time Horizon 1.1 100 min #13 of 26 Official leaderboard
WeirdML 43.7% #63 of 112 Official leaderboard
Internal model use survey (Anthropic) 0% #3 of 5 Model card
GSO-Bench 6.86% #19 of 30 Official leaderboard
LiveCodeBench v5 59.8% #14 of 21 Official leaderboard
LiveCodeBench v6 56.6% #14 of 21 Official leaderboard
SWE-bench Verified (hard subset) 39.5% #6 of 9 Model card
SWE-bench Verified 70.7% #24 of 32 Epoch AI
Quadruped reinforcement learning task (Anthropic) 1.25 #6 of 8 Model card
Text-based reinforcement learning task (Anthropic) 0.63 #6 of 7 Model card
Time series forecasting task (Anthropic) 6.15 #8 of 9 Model card
AlgoTune 1.33× #17 of 18 Official leaderboard
Internal AI research evaluation suite 2 0.35 #6 of 6 Model card

Agents and tools

BenchmarkScoreRankSource
tau-bench 70.5% #1 of 23 Model card
Terminal-Bench 50% #1 of 22 Model card
MCP-Universe 28.1% #9 of 30 Official leaderboard
Elimination Game 4 #23 of 56 Official leaderboard
LisanBench 598.0 #64 of 133 Official leaderboard
Step Race 2 #29 of 59 Official leaderboard
DeepResearch Bench 46.8% #12 of 24 Epoch AI
LLM Chess 29 #63 of 124 Official leaderboard
LMArena Search Arena 1,152 #33 of 35 Official leaderboard

Professional work

BenchmarkScoreRankSource
Vals Finance Agent v1 43.6% #14 of 47 Official leaderboard
LegalBench 83.1% #60 of 138 Official leaderboard
TaxEval v2 71.9% #68 of 129 Official leaderboard
MortgageTax 58.6% #65 of 86 Official leaderboard
Vending-Bench $2,077 #7 of 9 Official leaderboard
TaxCalcBench 27.4% #14 of 17 Official leaderboard

Vision, audio and robotics

BenchmarkScoreRankSource
MMMU 76.5% #17 of 136 Official leaderboard
VISTA 47% #19 of 54 Official leaderboard
EnigmaEval 5.57% #22 of 41 Epoch AI
Vision Arena 1,208 #74 of 136 Official leaderboard
ZeroBench 6.00% #37 of 66 Official leaderboard
VPCT 38% #20 of 28 Epoch AI
Drone-Bench 41% #14 of 19 Official leaderboard
Blueprint-Bench 0.35 #6 of 8 Official leaderboard
GeoBench 49% #43 of 48 Official leaderboard

Safety and security

BenchmarkScoreRankSource
BBQ (Bias Benchmark for QA) 91.1% #2 of 24 Model card
AIR-Bench 2024 85.7% #12 of 82 Official leaderboard
HELM Safety 96.9 #12 of 75 Official leaderboard
SimpleSafetyTests 100% #14 of 75 Official leaderboard
Bioweapons acquisition uplift trial (Anthropic) 2.53 #1 of 3 Model card
Average in-eval reward hacking rate (Anthropic) 15.2% #2 of 5 Model card
Cybench 38% #10 of 24 Official leaderboard
Single-turn benign request evaluations (Anthropic) 0.07% #8 of 18 Model card
Single-turn harmful request evaluations (Anthropic) 98.4% #8 of 18 Model card
FORTRESS 24.8% #32 of 66 Official leaderboard
Pwn CTF challenges (Anthropic) 55.6% #3 of 6 Model card
Anthropic Red Team 98.9% #41 of 75 Official leaderboard
Malicious use of Claude Code (Anthropic) 88% #9 of 16 Model card
Impossible tasks (Anthropic) 47% #7 of 11 Model card
Network CTF challenges (Anthropic) 50% #4 of 6 Model card
Rev CTF challenges (Anthropic) 50% #4 of 6 Model card
Bioweapons knowledge questions 51.5% #2 of 3 Model card
Computer use prompt injection evaluation (Anthropic) 71% #4 of 5 Model card
Crypto CTF challenges (Anthropic) 36.4% #5 of 6 Model card
Web CTF challenges (Anthropic) 80% #5 of 6 Model card
Creative biology 45% #6 of 7 Model card
Reward-hack-prone coding tasks (Anthropic) 9.00% #7 of 8 Model card
StrongREJECT 2.24% #20 of 22 Model card
SHADE-Arena 0% #16 of 16 Model card
Agentic misalignment suite (Anthropic) 84% #7 of 7 Model card
Many-shot jailbreaking evaluation (Anthropic) 25% #1 of 1 Model card

Other

BenchmarkScoreRankSource
HELM Capabilities 78.0 #8 of 60 Official leaderboard
LMCA 37.4% #62 of 139 Epoch AI
ACCoRD 65.8 #68 of 150 Official leaderboard
Conceptual Reasoning Index 49.5 #69 of 149 Official leaderboard

Anthropic's models over time

Anthropic's models in color, against every other model on the Epoch Capabilities Index, by release date.

50 75 100 125 150 175 2023 2024 2025 2026 2027 Claude Haiku 5.5 Claude Sonnet 5.5 Claude Sonnet 4.6 Claude Haiku 4.5 Claude 3.7 Sonnet Claude 3.5 Haiku Claude 3.5 Sonnet (October 2024) 50 75 100 125 150 175 2023 2024 2025 2026 2027 Claude Haiku 5.5 Claude Opus 4.5

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Its system card

Every result in the longer documents the lab published with it. 48 of them aren't in the launch post.