DeepSeek-R1

DeepSeek-R1 ranks first on 3 of the 76 benchmarks it has scores on.

DeepSeek Released 20 Jan 2025 Family DeepSeek-R1 The lab's model card

Benchmarks with a score

Ranked first, out of five or more models

In the top three

DeepSeek Jan 2025

DeepSeek-R1

Open weights Flagship

138.8 #131 of 271

Epoch Capabilities Index

CritPt 1.10% #94 /160
GPQA Diamond 71.7% #105 /219
AMO-Bench 10.9% #16 /21

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

Benchmark	Score	Rank	Source
MMLU	90.8%	#7 of 127	Model card
AlpacaEval 2.0	87.6%	#1 of 16	Model card
DROP	92.2%	#3 of 40	Model card
Vectara Hallucination Leaderboard (HHEM)	14.3%	#12 of 148	Official leaderboard
SEAL Precise Instruction Following	87.8	#2 of 18	Official leaderboard
SuperGPQA	61.8%	#7 of 58	Official leaderboard
LongBench v2	58.3%	#4 of 33	Official leaderboard
CLUEWSC	92.8%	#1 of 8	Model card
Arena-Hard-Auto	92.3%	#5 of 39	Model card
MMLU-Pro	85%	#25 of 151	Model card
BIG-Bench Extra Hard	34.9%	#2 of 11	Official leaderboard
LLM Confabulation Leaderboard	12.7%	#11 of 58	Official leaderboard
Lech Mazur Creative Writing	8.30	#9 of 44	Epoch AI
C-Eval	91.8%	#8 of 38	Model card
Fiction.LiveBench	69.4%	#17 of 57	Epoch AI
MMLU-Redux	93.4%	#12 of 38	Model card
SimpleQA	30.1%	#23 of 59	Model card
Arena-Hard v2.0	58%	#9 of 23	Official leaderboard
LMArena (Elo)	1,398	#132 of 333	Official leaderboard
AgentIF	57.9%	#5 of 12	Official leaderboard
FRAMES	83%	#4 of 8	Model card
WritingBench	72.1	#23 of 45	Official leaderboard
MultiNRC	24.3%	#24 of 38	Official leaderboard
IFEval	83.3%	#87 of 126	Model card
Humanity's Last Exam	17.7%	#109 of 147	Model card
Chinese SimpleQA	63.7%	#12 of 13	Model card
WideSearch	0.80%	#6 of 6	Official leaderboard

Math and science

Benchmark	Score	Rank	Source
ZebraLogic	78.7%	#3 of 46	Official leaderboard
PolyMath	47%	#5 of 38	Official leaderboard
U-MATH	80.7%	#4 of 30	Official leaderboard
MATH Level 5	93%	#14 of 94	Epoch AI
AIME 2024	91.4%	#14 of 87	Model card
MATH-500	97.3%	#17 of 67	Model card
Misguided Attention	55.1%	#13 of 45	Official leaderboard
CNMO 2024	86.9%	#2 of 6	Model card
USAMO 2025	4.76%	#4 of 10	Official leaderboard
AIME 2025	87.5%	#64 of 143	Model card
GPQA Diamond	71.7%	#105 of 219	Epoch AI
PutnamBench	0.15%	#8 of 14	Official leaderboard
CritPt	1.10%	#94 of 160	Epoch AI
OTIS Mock AIME 2024-2025	53.3%	#123 of 202	Epoch AI
HMMT 2025	79.4%	#35 of 49	Model card
AMO-Bench	10.9%	#16 of 21	Official leaderboard
SimpleBench	30.9%	#77 of 100	Official leaderboard
ARC-AGI-2	1.30%	#77 of 95	Official leaderboard
BRUMO 2025	80.8%	#36 of 42	Official leaderboard
ARC-AGI	15.8%	#80 of 92	Official leaderboard
SMT 2025	67%	#36 of 40	Official leaderboard
AMC 12 2024	143.7	#1 of 1	Model card

Coding

Benchmark	Score	Rank	Source
BigCodeBench-Hard	35.1%	#4 of 96	Official leaderboard
BIRD-CRITIC	32%	#2 of 9	Official leaderboard
SEAL Coding	1,100	#7 of 26	Official leaderboard
SciCode	4.60%	#5 of 18	Official leaderboard
LiveCodeBench	73.3%	#22 of 73	Model card
SWE-PolyBench	11.5%	#2 of 6	Official leaderboard
Aider Polyglot	56.9%	#21 of 54	Epoch AI
AlgoTune	1.70×	#7 of 18	Epoch AI
METR Time Horizon	28 min	#16 of 31	Official leaderboard
Multi-SWE-bench	5.88%	#6 of 11	Official leaderboard
Codeforces rating	2,029	#25 of 41	Model card
SWE-bench Verified	57.6%	#112 of 150	Model card
WeirdML	36.5%	#91 of 112	Official leaderboard

Agents and tools

Benchmark	Score	Rank	Source
ToolComp	65.3%	#5 of 32	Official leaderboard
Step Race	3	#19 of 59	Official leaderboard
BALROG	34.9%	#12 of 35	Official leaderboard
Elimination Game	4	#30 of 56	Official leaderboard
tau-bench	58.7%	#13 of 23	Model card
CRMArena	35.1%	#6 of 9	Official leaderboard
LLM Chess	-216	#96 of 124	Official leaderboard
Berkeley Function Calling Leaderboard (BFCL)	37%	#47 of 47	Model card

Professional work

Benchmark	Score		Rank	Source
xbench-Recruitment	48.3		#6 of 8	Official leaderboard

Safety and security

Benchmark	Score	Rank	Source
BBQ (Bias Benchmark for QA)	96.6%	#1 of 24	Model card
HarmBench	35%	#3 of 4	Model card
XSTest	95.3%	#14 of 16	Model card
FORTRESS	74.4%	#66 of 66	Official leaderboard
Do-Not-Answer	94.8%	#2 of 2	Model card
Anthropic Red Team	96.2%	#1 of 1	Model card
DeepSeek in-house safety benchmark	25.2%	#1 of 1	Model card
DeepSeek jailbreak robustness evaluation	85.9%	#1 of 1	Model card
DeepSeek multilingual safety evaluation	74.2%	#1 of 1	Model card
SimpleSafetyTests	97.5%	#1 of 1	Model card

Other

Benchmark	Score	Rank	Source
LiveBench	71.6%	#7 of 48	Epoch AI
CAIS Risk Index	66.9	#60 of 65	Official leaderboard
CAIS Text Capabilities Index	11.8	#61 of 66	Official leaderboard

DeepSeek's models over time

DeepSeek's models in color, against every other model on the Epoch Capabilities Index, by release date.

Its launch posts

The benchmarks the lab chose to headline, with the scores it gave.

DeepSeek-R1 model card 20 Jan 2025

DeepSeek-R1-0528 model card 28 May 2025

MMLU-Redux (EM) 93.4
MMLU-Pro (EM) 85
GPQA-Diamond (Pass@1) 81
SimpleQA (Correct) 27.8
FRAMES (Acc.) 83
Humanity's Last Exam (Pass@1) 17.7
LiveCodeBench (2408-2505) (Pass@1) 73.3
Codeforces-Div1 (Rating) 1930
SWE Verified (Resolved) 57.6
Aider-Polyglot (Acc.) 71.6
AIME 2024 (Pass@1) 91.4
AIME 2025 (Pass@1) 87.5
HMMT 2025 (Pass@1) 79.4
CNMO 2024 (Pass@1) 86.9
BFCL_v3_MultiTurn (Acc) 37
Mean of Tau-Bench (Pass@1) Airline / Tau-Bench (Pass@1) Retail 58.7

Its system card

Every result in the longer documents the lab published with it. 11 of them aren't in the launch post.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (arXiv 2501.12948) 20 Jan 2025

Capabilities