Qwen2.5-Coder (7B)
Qwen2.5-Coder (7B).
Qwen Released 18 Sep 2024 10
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
Qwen Sep 2024 Qwen2.5-Coder (7B)
Open weights
113.1
#219 of 271
Epoch Capabilities Index
- LiveSQLBench 6.33% #34 /36
- BigCodeBench-Hard 20.3% #58 /96
- AutoCodeBench 22.5% #31 /34
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ARC-Challenge | 60.9% | #33 of 75 | Epoch AI | |
| HellaSwag | 76.8% | #38 of 74 | Epoch AI | |
| MMLU | 68% | #68 of 127 | Epoch AI | |
| WinoGrande | 72.9% | #42 of 76 | Epoch AI |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 86.7% | #12 of 82 | Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench | 44.6% | #37 of 69 | Official leaderboard | |
| BigCodeBench-Hard | 20.3% | #58 of 96 | Official leaderboard | |
| AutoCodeBench | 22.5% | #31 of 34 | Official leaderboard | |
| LiveSQLBench | 6.33% | #34 of 36 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AceBench | 48.9% | #13 of 16 | Official leaderboard |
Qwen's models over time
Qwen's models in color, against every other model on the Epoch Capabilities Index, by release date.