Mistral Medium 3

Mistral Medium 3 ranks first on Arena-Hard-Auto.

Mistral AI Released 7 May 2025 Family Mistral Medium The lab's model card

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Mistral AI May 2025

Closed weights Mid-size

134.0 #150 of 271

Epoch Capabilities Index

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
Arena-Hard-Auto	97.1%	#1 of 39	Model card
RULER	96%	#3 of 38	Model card
IFEval	89.4%	#33 of 126	Model card
LMArena (Elo)	1,387	#147 of 333	Official leaderboard
Lech Mazur Creative Writing	7.73	#21 of 44	Epoch AI
MMLU-Pro	77.2%	#92 of 151	Model card
LLM Confabulation Leaderboard	21.9%	#43 of 58	Official leaderboard
Judgemark v2.1	58.5	#32 of 42	Official leaderboard
Humanity's Last Exam	4.52%	#43 of 47	Official leaderboard
Global MMLU Lite	55.1%	#43 of 43	Official leaderboard

Benchmark	Score	Rank	Source
MATH Level 5	81.6%	#28 of 94	Epoch AI
Misguided Attention	50.6%	#18 of 45	Official leaderboard
GPQA Diamond	59.5%	#128 of 219	Epoch AI
HiPhO	40.4%	#23 of 35	Official leaderboard
OTIS Mock AIME 2024-2025	32.2%	#133 of 202	Epoch AI
DTBench	37.1%	#132 of 178	Official leaderboard
MATH-500	91%	#50 of 67	Model card
FrontierMath	0.35%	#64 of 70	Epoch AI

Benchmark	Score	Rank	Source
HumanEval	92.1%	#7 of 95	Model card
MultiPL-E	81.4%	#3 of 18	Model card
LiveCodeBench v6	30.3%	#75 of 76	Model card

Benchmark	Score	Rank	Source
BFCL V4	37.7%	#30 of 65	Official leaderboard
LisanBench	130.7	#108 of 133	Official leaderboard
Elimination Game	2	#52 of 56	Official leaderboard

Benchmark	Score	Rank	Source
DocVQA	95.3%	#8 of 33	Model card
AI2D	93.7%	#11 of 44	Model card
ChartQA	82.6%	#24 of 37	Model card
Vision Arena	1,155	#98 of 136	Official leaderboard
MMMU	66.1%	#71 of 97	Model card
VISTA	34.6%	#42 of 54	Official leaderboard

Benchmark	Score		Rank	Source
Human Pathogen Capabilities Test v2 (HPCT v2)	24.7%		#69 of 71	Official leaderboard

Mistral AI's models in color, against every other model on the Epoch Capabilities Index, by release date.

The benchmarks the lab chose to headline, with the scores it gave.