Benchmark Almanac

GLM-4

GLM-4 ranks first on Needle In A Haystack.

Z.ai Released 17 Jan 2024 Family GLM-4 The lab's model card
15
Benchmarks with a score
1
Ranked first, out of five or more models
1
In the top three
Z.ai Jan 2024

GLM-4

Closed weights Flagship

No Epoch Capabilities Index score yet
  • MathBench 51.3% #10 /21
  • IFEval 67.7% #118 /126
  • MMLU 81.5% #65 /127

Where it ranks

Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.

Knowledge and language

BenchmarkScoreRankSource
Needle In A Haystack 100% #1 of 9 Model card
BIG-Bench Hard 82.3% #30 of 71 Model card
HellaSwag 85.4% #22 of 47 Model card
LongBench 81.1% #1 of 2 Model card
MMLU 81.5% #65 of 127 Model card
AlignBench 7.88 #12 of 15 Model card
IFEval 67.7% #118 of 126 Model card
GLM-4 All Tools information seeking (Z.ai) 78.1% #1 of 1 Model card

Math and science

BenchmarkScoreRankSource
Math23K 88.5% #1 of 3 Model card
MathBench 51.3% #10 of 21 Official leaderboard
GSM8K 87.6% #47 of 87 Model card
MATH 47.9% #68 of 93 Model card
GPQA Diamond 35.7% #267 of 284 Model card

Coding

BenchmarkScoreRankSource
HumanEval 72% #65 of 95 Model card
CodeEditorBench 57.2% #6 of 7 Official leaderboard

Its launch post

The benchmarks the lab chose to headline, with the scores it gave.

Its system card

Every result in the longer documents the lab published with it. 3 of them aren't in the launch post.