Qwen2.5-7B

Qwen2.5-7B places in the top three on MadinahQA.

Qwen Released 19 Sep 2024

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Qwen Sep 2024

Open weights

118.5 #204 of 271

Epoch Capabilities Index

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
MadinahQA	62.7%	#3 of 39	Official leaderboard
ArabicMMLU	61.4%	#9 of 39	Official leaderboard
AlGhafa	72.2%	#10 of 39	Official leaderboard
Open Arabic LLM Leaderboard v2	61.1%	#11 of 39	Official leaderboard
ArbMMLU-HT	51.7%	#12 of 39	Official leaderboard
ALRAGE	77.4%	#13 of 39	Official leaderboard
MMLU	72.9%	#48 of 127	Epoch AI
MMLU-CF	61.3%	#17 of 37	Official leaderboard
MT-Bench	8.75	#12 of 23	Model card
EQ-Bench v2	69.2	#54 of 85	Official leaderboard
Chinese SimpleQA	27.9%	#19 of 27	Official leaderboard
RULER	85.4%	#27 of 38	Model card
SuperGPQA	28.8%	#43 of 58	Official leaderboard
Vectara Hallucination Leaderboard (HHEM)	2.80%	#111 of 148	Official leaderboard
MGSM	66.1%	#40 of 52	Model card
LongBench v2	35.6%	#27 of 33	Official leaderboard
FaithJudge	38.5%	#31 of 37	Official leaderboard
Arena-Hard-Auto	52%	#33 of 39	Model card
WritingBench	49.5	#39 of 45	Official leaderboard
Okapi MMLU	67%	#7 of 8	Model card
MMLU-Pro	56.3%	#134 of 151	Model card
IFEval	71.2%	#115 of 126	Model card
BuzzBench	31.4	#21 of 23	Official leaderboard
HELM Arabic	66.0	#26 of 28	Official leaderboard
MMLU-Redux	75.4%	#36 of 38	Model card
KMMLU	46.6%	#6 of 6	Model card
LongBench-Chat	7.42	#6 of 6	Model card
BLEnD	23.7%	#5 of 5	Model card
AMMLU	59.8%	#4 of 4	Model card
IndoMMLU	56.4%	#4 of 4	Model card
JMMLU	61.9%	#4 of 4	Model card
Multilingual IFEval	74.9%	#4 of 4	Model card
TurkishMMLU	54.3%	#4 of 4	Model card

Benchmark	Score	Rank	Source
MATH	75.5%	#32 of 93	Model card
GSM8K	91.6%	#32 of 87	Model card
KOR-Bench	38.6%	#22 of 33	Official leaderboard
U-MATH	43.3%	#20 of 30	Official leaderboard
ZebraLogic	12%	#31 of 46	Official leaderboard
GPQA Diamond	35.5%	#186 of 219	Epoch AI
OTIS Mock AIME 2024-2025	2.50%	#184 of 202	Epoch AI
DTBench	12.9%	#171 of 178	Official leaderboard

Benchmark	Score	Rank	Source
MBPP	79.2%	#24 of 66	Model card
HumanEval	84.8%	#41 of 95	Model card
BigCodeBench	41.9%	#45 of 69	Official leaderboard
MultiPL-E	70.4%	#13 of 18	Model card
BigCodeBench-Hard	14.2%	#72 of 96	Official leaderboard
LiveCodeBench	28.7%	#63 of 73	Model card

Benchmark	Score		Rank	Source
AceBench	54.8%		#11 of 16	Official leaderboard
BALROG	7.80%		#33 of 35	Official leaderboard

Benchmark	Score		Rank	Source
LegalBench	69.6%		#117 of 138	Official leaderboard
LEXam	16.7%		#29 of 31	Official leaderboard

Benchmark	Score	Rank	Source
AraTrust	80.7%	#13 of 39	Official leaderboard
Anthropic Red Team	98.5%	#48 of 75	Official leaderboard
HELM Safety	89.9	#49 of 75	Official leaderboard
SimpleSafetyTests	96%	#62 of 75	Official leaderboard
AIR-Bench 2024	47%	#68 of 82	Official leaderboard
BaxBench	2.22%	#31 of 34	Official leaderboard

Benchmark	Score	Rank	Source
HELM Lite	48.8	#41 of 71	Official leaderboard
HELM Capabilities	52.9	#52 of 60	Official leaderboard
ACCoRD	27.3	#142 of 150	Official leaderboard
LMCA	6.42%	#133 of 139	Epoch AI
Conceptual Reasoning Index	11.9	#143 of 149	Official leaderboard
LiveBench	35.9%	#26 of 27	Model card

Qwen's models in color, against every other model on the Epoch Capabilities Index, by release date.

Every result in the longer documents the lab published with it. 27 of them aren't in the launch post.