Qwen2.5-32B

Qwen2.5-32B places in the top three on 4 benchmarks.

Qwen Released 17 Sep 2024

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Qwen Sep 2024

Open weights

No Epoch Capabilities Index score yet

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
AlGhafa	78.1%	#3 of 39	Official leaderboard
MT-Bench	9.20	#2 of 23	Model card
Open Arabic LLM Leaderboard v2	70%	#4 of 39	Official leaderboard
MMLU-CF	69.7%	#4 of 37	Official leaderboard
ArabicMMLU	68%	#5 of 39	Official leaderboard
ArbMMLU-HT	65.7%	#5 of 39	Official leaderboard
MadinahQA	57.1%	#7 of 39	Official leaderboard
ALRAGE	79.2%	#8 of 39	Official leaderboard
EQ-Bench v2	79.9	#25 of 85	Official leaderboard
RULER	92.9%	#13 of 38	Model card
MGSM	87.2%	#18 of 52	Model card
Chinese SimpleQA	41.1%	#12 of 27	Official leaderboard
Okapi MMLU	77.2%	#4 of 8	Model card
LongBench-Chat	8.70	#3 of 6	Model card
AMMLU	70.4%	#2 of 4	Model card
IndoMMLU	66.4%	#2 of 4	Model card
JMMLU	76.5%	#2 of 4	Model card
Multilingual IFEval	82.7%	#2 of 4	Model card
TurkishMMLU	72.4%	#2 of 4	Model card
FaithJudge	19.2%	#21 of 37	Official leaderboard
Arena-Hard-Auto	74.5%	#23 of 39	Model card
BLEnD	27.9%	#3 of 5	Model card
SuperGPQA	38.8%	#37 of 58	Official leaderboard
KMMLU	60.8%	#4 of 6	Model card
Vectara Hallucination Leaderboard (HHEM)	3.00%	#104 of 148	Official leaderboard
MMLU-Pro	69%	#114 of 151	Model card
IFEval	79.5%	#99 of 126	Model card
MMLU-Redux	83.9%	#31 of 38	Model card

Benchmark	Score	Rank	Source
GSM8K	95.9%	#7 of 87	Model card
MATH	83.1%	#16 of 93	Model card
KOR-Bench	54.7%	#6 of 33	Official leaderboard
ZebraLogic	26.1%	#17 of 46	Official leaderboard
U-MATH	52.4%	#14 of 30	Official leaderboard
MATH Level 5	56.1%	#46 of 94	Epoch AI
GPQA Diamond	46.1%	#163 of 219	Epoch AI
OTIS Mock AIME 2024-2025	7.36%	#160 of 202	Epoch AI

Benchmark	Score	Rank	Source
MBPP	84%	#10 of 66	Model card
HumanEval	88.4%	#22 of 95	Model card
BigCodeBench	48.6%	#27 of 69	Official leaderboard
MultiPL-E	75.4%	#9 of 18	Model card
BigCodeBench-Hard	24.6%	#49 of 96	Official leaderboard
LiveCodeBench	51.2%	#43 of 73	Model card
Aider-Edit	54.1%	#31 of 37	Official leaderboard

Benchmark	Score		Rank	Source
AgentBench FC	37.2%		#10 of 12	Official leaderboard

Benchmark	Score		Rank	Source
AraTrust	86.7%		#7 of 39	Official leaderboard

Benchmark	Score		Rank	Source
Political Science LLM Benchmark	64.9%		#18 of 29	Official leaderboard
LiveBench	50.7%		#20 of 27	Model card

Every result in the longer documents the lab published with it. 27 of them aren't in the launch post.