DeepSeek-V3

DeepSeek-V3 ranks first on 3 of the 78 benchmarks it has scores on.

DeepSeek Released 26 Dec 2024 Family DeepSeek-V3 The lab's model card

Benchmarks with a score

Ranked first, out of five or more models

In the top three

DeepSeek Dec 2024

Open weights Flagship

132.3 #156 of 271

Epoch Capabilities Index

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
ARC-Easy	98.9%	#1 of 23	Model card
ARC-Challenge	95.3%	#4 of 45	Model card
DROP	91.6%	#4 of 40	Model card
BIG-Bench Hard	87.5%	#12 of 71	Model card
AGIEval	79.6%	#6 of 35	Model card
MMLU	88.5%	#23 of 127	Model card
AlpacaEval 2.0	70%	#3 of 16	Model card
WinoGrande	84.9%	#8 of 42	Model card
HellaSwag	88.9%	#9 of 47	Model card
Natural Questions	40%	#4 of 20	Model card
PIQA	84.7%	#7 of 33	Model card
TriviaQA	82.9%	#13 of 39	Model card
CLUEWSC	90.9%	#3 of 8	Model card
MMLU-Pro	81.2%	#65 of 151	Model card
Arena-Hard-Auto	85.5%	#17 of 39	Model card
SuperGPQA	47.4%	#26 of 58	Official leaderboard
CCPM	92%	#1 of 2	Model card
RewardBench	87%	#1 of 2	Model card
LMArena (Elo)	1,358	#168 of 333	Official leaderboard
IFEval	86.1%	#64 of 126	Model card
SimpleQA	24.9%	#30 of 59	Model card
CMMLU	88.8%	#9 of 17	Model card
MGSM	79.8%	#31 of 52	Model card
Vectara Hallucination Leaderboard (HHEM)	3.90%	#89 of 148	Official leaderboard
SEAL Precise Instruction Following	82.3	#11 of 18	Official leaderboard
MMLU-Redux	89.1%	#24 of 38	Model card
C-Eval	86.5%	#25 of 38	Model card
MMMLU	79.4%	#41 of 62	Model card
C3	78.6%	#4 of 6	Model card
WritingBench	59.2	#31 of 45	Official leaderboard
LongBench v2	48.7%	#21 of 27	Model card
Chinese SimpleQA	64.8%	#11 of 13	Model card
FRAMES	73.3%	#8 of 8	Model card
RACE-H	51.3%	#7 of 7	Model card
RACE-M	67.1%	#3 of 3	Model card
CMRC	76.3%	#1 of 1	Model card

Benchmark	Score	Rank	Source
ZebraLogic	42.1%	#6 of 46	Official leaderboard
U-MATH	62.6%	#8 of 30	Official leaderboard
MATH Level 5	64.8%	#39 of 94	Epoch AI
ForecastBench	62.6	#35 of 83	Epoch AI
GSM8K	89.3%	#40 of 87	Model card
CMATH	90.7%	#3 of 6	Model card
MATH	61.6%	#50 of 93	Model card
GPQA Diamond	56.5%	#132 of 219	Epoch AI
OTIS Mock AIME 2024-2025	15.8%	#149 of 202	Epoch AI
CritPt	0%	#122 of 160	Epoch AI
MATH-500	90.2%	#52 of 67	Model card
AIME 2024	59.4%	#69 of 87	Model card
FrontierMath	1.72%	#56 of 70	Epoch AI
PolyMath	20.4%	#32 of 38	Official leaderboard
Misguided Attention	26.9%	#39 of 45	Official leaderboard
SimpleBench	18.9%	#95 of 100	Official leaderboard
CNMO 2024	43.2%	#6 of 6	Model card

Benchmark	Score	Rank	Source
BigCodeBench	56.1%	#2 of 69	Official leaderboard
BigCodeBench-Hard	34.5%	#5 of 96	Official leaderboard
HumanEval+	86.6%	#6 of 46	Official leaderboard
Aider-Edit	79.7%	#1 of 5	Model card
HumanEval Multi	82.6%	#1 of 5	Model card
MBPP+	73%	#10 of 35	Official leaderboard
SciCode	3.10%	#7 of 18	Official leaderboard
MBPP	75.4%	#27 of 66	Model card
CRUXEval	69.8%	#4 of 9	Model card
Aider Polyglot	48.4%	#28 of 54	Epoch AI
SWE-bench Lite	30.7%	#9 of 15	Official leaderboard
METR Time Horizon	20 min	#19 of 31	Official leaderboard
LiveCodeBench	49.2%	#45 of 73	Model card
SEAL Coding	985	#17 of 26	Official leaderboard
Multi-SWE-bench	4.72%	#8 of 11	Official leaderboard
HumanEval	65.2%	#71 of 95	Model card
LiveCodeBench v5	31.7%	#17 of 21	Official leaderboard
BigO(Bench)	27.1%	#16 of 18	Official leaderboard
SWE-bench Verified	42%	#136 of 150	Model card
LiveCodeBench v6	27.2%	#20 of 21	Official leaderboard

Benchmark	Score	Rank	Source
AceBench	74.8%	#5 of 16	Official leaderboard
ToolComp	62.5%	#12 of 32	Official leaderboard
Elimination Game	4	#29 of 56	Official leaderboard
LisanBench	372.3	#74 of 133	Official leaderboard
Step Race	1	#46 of 59	Official leaderboard
Factorio Learning Environment	48585.0	#5 of 6	Official leaderboard
LLM Chess	-823	#124 of 124	Official leaderboard

Benchmark	Score		Rank	Source
BaxBench	24.7%		#20 of 34	Official leaderboard

Benchmark	Score		Rank	Source
LiveBench	60.5%		#15 of 48	Epoch AI
Pile-test	0.55		#1 of 1	Model card

DeepSeek's models in color, against every other model on the Epoch Capabilities Index, by release date.

The benchmarks the lab chose to headline, with the scores it gave.

Every result in the longer documents the lab published with it. 1 of them isn't in the launch post.