Qwen1.5-72B

Qwen1.5-72B.

Qwen Released 4 Feb 2024 Family Qwen1.5 The lab's model card

Benchmarks with a score

Ranked first, out of five or more models

In the top three

Qwen Feb 2024

Open weights Flagship

No Epoch Capabilities Index score yet

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Benchmark	Score	Rank	Source
EQ-Bench v2	82.8	#12 of 85	Official leaderboard
MixEval	84.1%	#10 of 34	Official leaderboard
WildBench	43.5	#22 of 47	Official leaderboard
MMLU-CF	59.8%	#18 of 37	Official leaderboard
RepoQA	67%	#12 of 22	Official leaderboard
ThaiExam	53.8%	#14 of 24	Official leaderboard
MMLU-Pro	52.6%	#129 of 212	Official leaderboard
CMMLU	83.5%	#11 of 17	Model card
MT-Bench	8.61	#15 of 23	Model card
MMLU	77.5%	#84 of 127	Model card
C-Eval	84.1%	#27 of 38	Model card
BIG-Bench Hard	65.5%	#51 of 71	Model card
LMArena (Elo)	1,234	#268 of 333	Official leaderboard
MGSM	61.7%	#44 of 52	Model card
Flores	35.6	#7 of 8	Model card
AlpacaEval 2.0	27.2%	#15 of 16	Model card
RewardBench	67.2%	#46 of 48	Official leaderboard
Multilingual Exams	66.3%	#3 of 3	Model card
Multilingual Understanding	78.2%	#3 of 3	Model card
L-Eval	78.1%	#1 of 1	Model card

Benchmark	Score	Rank	Source
GSM8K	79.5%	#61 of 87	Model card
MATH	34.1%	#78 of 93	Model card
GPQA Diamond	28.8%	#200 of 219	Epoch AI

Benchmark	Score	Rank	Source
MBPP+	61.6%	#19 of 35	Official leaderboard
HumanEval+	59.1%	#27 of 46	Official leaderboard
BigCodeBench	36.8%	#51 of 69	Official leaderboard
MBPP	53.4%	#54 of 66	Model card
BigCodeBench-Hard	12.2%	#79 of 96	Official leaderboard
HumanEval	41.5%	#83 of 95	Model card

Benchmark	Score	Rank	Source
GTA	13.3%	#6 of 15	Official leaderboard
Code Interpreter Benchmark	73.1%	#2 of 4	Model card
T-Eval	76.7%	#1 of 2	Model card
Chinese Tool-Use Benchmark	95.1%	#3 of 4	Model card

Benchmark	Score	Rank	Source
SimpleSafetyTests	99%	#31 of 75	Official leaderboard
OR-Bench	46.9%	#14 of 28	Official leaderboard
Anthropic Red Team	99%	#40 of 75	Official leaderboard
HELM Safety	88.6	#54 of 75	Official leaderboard
AIR-Bench 2024	48.6%	#67 of 82	Official leaderboard

Benchmark	Score		Rank	Source
HELM Lite	60.8		#28 of 71	Official leaderboard

The benchmarks the lab chose to headline, with the scores it gave.