Qwen-7B.
Qwen Released 28 Sep 2023 Qwen Sep 2023 Open weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ARC-Challenge | 75.3% | #21 of 75 | Epoch AI | |
| BoolQ | 76.4% | #47 of 75 | Epoch AI | |
| PIQA | 77.9% | #42 of 58 | Epoch AI | |
| LAMBADA | 67.9% | #23 of 26 | Epoch AI | |
| BIG-Bench Hard | 46.7% | #63 of 71 | Model card | |
| EQ-Bench v2 | 50.1 | #77 of 85 | Official leaderboard | |
| MMLU | 57% | #116 of 127 | Model card | |
| C-Eval | 59.7% | #37 of 38 | Model card | |
| Qwen Chinese human evaluation | 28.4% | #2 of 2 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 37.2% | #87 of 95 | Model card | |
| MBPP | 35.8% | #61 of 66 | Model card | |
| HumanEvalPack | 24.2% | #3 of 3 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Chinese Tool-Use Benchmark | 98% | #2 of 4 | Model card | |
| HuggingFace Agent Benchmark | 87% | #2 of 3 | Model card | |
| Code Interpreter Benchmark | 70.2% | #3 of 4 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SuperCLUE-Safety | 78.6 | #6 of 7 | Official leaderboard |
Qwen's models in color, against every other model on the Epoch Capabilities Index, by release date.
Every result in the longer documents the lab published with it. 14 of them aren't in the launch post.