Qwen2.5-14B

Qwen2.5-14B.

Qwen Released 19 Sep 2024

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Qwen Sep 2024

Open weights

No Epoch Capabilities Index score yet

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
ArabicMMLU	68.5%	#4 of 39	Official leaderboard
MadinahQA	59.9%	#5 of 39	Official leaderboard
AlGhafa	77.2%	#6 of 39	Official leaderboard
ArbMMLU-HT	61.7%	#6 of 39	Official leaderboard
Open Arabic LLM Leaderboard v2	68.5%	#6 of 39	Official leaderboard
MMLU	79.9%	#24 of 127	Epoch AI
MMLU-CF	66.4%	#9 of 37	Official leaderboard
ALRAGE	78.9%	#10 of 39	Official leaderboard
EQ-Bench v2	79.2	#26 of 85	Official leaderboard
MT-Bench	8.88	#10 of 23	Model card
RULER	91.4%	#17 of 38	Model card
MGSM	82.3%	#28 of 52	Model card
Vectara Hallucination Leaderboard (HHEM)	4.20%	#80 of 148	Official leaderboard
Chinese SimpleQA	37.2%	#15 of 27	Official leaderboard
Okapi MMLU	72.1%	#5 of 8	Model card
SuperGPQA	35.1%	#40 of 58	Official leaderboard
Arena-Hard-Auto	68.3%	#28 of 39	Model card
AMMLU	66.8%	#3 of 4	Model card
IndoMMLU	65.1%	#3 of 4	Model card
JMMLU	72.8%	#3 of 4	Model card
Multilingual IFEval	77.1%	#3 of 4	Model card
TurkishMMLU	66.8%	#3 of 4	Model card
IFEval	81%	#95 of 126	Model card
FaithJudge	29%	#28 of 37	Official leaderboard
BLEnD	27%	#4 of 5	Model card
KMMLU	59.7%	#5 of 6	Model card
LongBench-Chat	8.04	#5 of 6	Model card
MMLU-Pro	63.7%	#127 of 151	Model card
MMLU-Redux	80%	#35 of 38	Model card

Benchmark	Score	Rank	Source
GSM8K	94.8%	#14 of 87	Model card
MATH	80%	#22 of 93	Model card
KOR-Bench	49.4%	#12 of 33	Official leaderboard
GPQA Diamond	45.5%	#243 of 284	Model card

Benchmark	Score	Rank	Source
MBPP	82%	#14 of 66	Model card
BigCodeBench	46%	#32 of 69	Official leaderboard
HumanEval	83.5%	#45 of 95	Model card
BigCodeBench-Hard	20.9%	#54 of 96	Official leaderboard
LiveCodeBench	42.6%	#48 of 73	Model card
MultiPL-E	72.8%	#12 of 18	Model card

Benchmark	Score		Rank	Source
AgentBench FC	27.2%		#12 of 12	Official leaderboard

Benchmark	Score		Rank	Source
AraTrust	86.5%		#8 of 39	Official leaderboard

Benchmark	Score		Rank	Source
LiveBench	44.4%		#24 of 27	Model card

Every result in the longer documents the lab published with it. 27 of them aren't in the launch post.