ChatGLM3-6B
ChatGLM3-6B places in the top three on 2 benchmarks.
22
Benchmarks with a score
0
Ranked first, out of five or more models
2
In the top three
Z.ai Oct 2023
ChatGLM3-6B
Open weights Open model
No Epoch Capabilities Index score yet
- MathBench 19.9% #20 /21
- MMLU 61.4% #112 /127
- C-Eval 69% #34 /38
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BoolQ | 87.9% | #2 of 28 | Model card | |
| CommonSenseQA | 86.5% | #2 of 17 | Model card | |
| AGIEval | 53.7% | #19 of 35 | Model card | |
| BIG-Bench Hard | 66.1% | #50 of 71 | Model card | |
| HellaSwag | 79.7% | #34 of 47 | Model card | |
| PIQA | 80.1% | #26 of 33 | Model card | |
| DROP | 70.9% | #35 of 40 | Model card | |
| CriticEval | 3.95 | #7 of 8 | Official leaderboard | |
| MMLU | 61.4% | #112 of 127 | Model card | |
| C-Eval | 69% | #34 of 38 | Model card | |
| Chinese SimpleQA | 12% | #26 of 27 | Official leaderboard | |
| LMArena (Elo) | 1,057 | #327 of 333 | Official leaderboard | |
| CMMLU | 67.5% | #17 of 17 | Model card | |
| C3 | 73.9% | #6 of 6 | Model card | |
| GAOKAO-Bench | 67.3% | #2 of 2 | Model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 72.3% | #67 of 87 | Model card | |
| MATH | 25.7% | #83 of 93 | Model card | |
| MathBench | 19.9% | #20 of 21 | Official leaderboard | |
| GPQA Diamond | 26.8% | #282 of 284 | Model card |
Coding
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | 57.9% | #42 of 47 | Model card |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 10 of them aren't in the launch post.
Capabilities
- GSM8K 72.3
- MATH 25.7
- BIG-Bench Hard 66.1
- MMLU 61.4
- GPQA Diamond 26.8
- HumanEval 58.5
- BoolQ 87.9
- CommonSenseQA 86.5
- HellaSwag 79.7
- PIQA 80.1
- DROP 70.9
- C-Eval 69
- CMMLU 67.5
- GAOKAO-Bench 67.3
- C3 73.9
- Berkeley Function Calling Leaderboard (BFCL) 57.9