Mistral Small 3

Mistral Small 3.

Mistral AI Released 25 Jan 2025 Family Mistral Small The lab's model card

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Mistral AI Jan 2025

Open weights Small

No Epoch Capabilities Index score yet

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
Vectara Hallucination Leaderboard (2025 dataset)	5.10%	#11 of 104	Official leaderboard
ARC-Challenge	91.3%	#13 of 45	Model card
AGIEval	65.8%	#12 of 35	Model card
Arena-Hard-Auto	87.3%	#14 of 39	Model card
TriviaQA	80.3%	#17 of 39	Model card
FaithJudge	17%	#17 of 37	Official leaderboard
MMLU	80.7%	#70 of 127	Model card
Vectara Hallucination Leaderboard (HHEM)	3.10%	#102 of 148	Official leaderboard
IFEval	82.9%	#88 of 126	Model card
LMArena (Elo)	1,274	#249 of 333	Official leaderboard
WildBench	52.3	#3 of 4	Model card
MMLU-Pro	66.3%	#121 of 151	Model card
MT-Bench	8.35	#19 of 23	Model card
LLM Confabulation Leaderboard	25.2%	#48 of 58	Official leaderboard
EQ-Bench 3	862	#71 of 78	Official leaderboard
Creative Writing v3	42.4	#117 of 120	Official leaderboard

Benchmark	Score	Rank	Source
MATH	70.6%	#40 of 93	Model card
MATH Level 5	44.8%	#58 of 94	Epoch AI
GSM8K	80.7%	#59 of 87	Model card
GPQA Diamond	47.3%	#159 of 219	Epoch AI
OTIS Mock AIME 2024-2025	6.67%	#164 of 202	Epoch AI
U-MATH	34.8%	#25 of 30	Official leaderboard
Misguided Attention	27.9%	#38 of 45	Official leaderboard

Benchmark	Score	Rank	Source
BigCodeBench	47.8%	#29 of 69	Official leaderboard
HumanEval	84.8%	#42 of 95	Model card
BigCodeBench-Hard	20.6%	#57 of 96	Official leaderboard
MBPP	69.6%	#41 of 66	Model card

Benchmark	Score		Rank	Source
SnakeBench	25		#77 of 183	Official leaderboard
Elimination Game	3		#41 of 56	Official leaderboard

Benchmark	Score	Rank	Source
Anthropic Red Team	96.4%	#66 of 75	Official leaderboard
HELM Safety	84.7	#66 of 75	Official leaderboard
SimpleSafetyTests	93.3%	#66 of 75	Official leaderboard
AIR-Bench 2024	32.8%	#78 of 82	Official leaderboard

Benchmark	Score		Rank	Source
LiveBench	42.5%		#36 of 48	Epoch AI

The benchmarks the lab chose to headline, with the scores it gave.