Qwen2.5-Coder-32B-Instruct
Qwen2.5-Coder-32B-Instruct places in the top three on 2 benchmarks.
Qwen Released 21 Nov 2024 15
Benchmarks with a score
0
Ranked first, out of five or more models
2
In the top three
Qwen Nov 2024 Qwen2.5-Coder-32B-Instruct
Open weights
No Epoch Capabilities Index score yet
- BigCodeBench-Hard 30.8% #18 /96
- BigO(Bench) 34.9% #9 /18
- BIRD-CRITIC 20% #9 /9
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LMArena (Elo) | 1,271 | #251 of 333 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 93% | #3 of 82 | Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP+ | 77% | #3 of 35 | Official leaderboard | |
| BigCodeBench | 53.5% | #6 of 69 | Official leaderboard | |
| HumanEval+ | 87.2% | #4 of 46 | Official leaderboard | |
| BigCodeBench-Hard | 30.8% | #18 of 96 | Official leaderboard | |
| Aider-Edit | 71.4% | #11 of 37 | Official leaderboard | |
| BigO(Bench) | 34.9% | #9 of 18 | Official leaderboard | |
| SWE-bench Verified | 38% | #46 of 60 | Official leaderboard | |
| Aider Polyglot | 16.4% | #46 of 54 | Epoch AI | |
| BIRD-CRITIC | 20% | #9 of 9 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AceBench | 79.6% | #4 of 16 | Official leaderboard | |
| SnakeBench | 20 | #130 of 183 | Official leaderboard | |
| GAIA | 0.33% | #56 of 57 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LiveBench | 46.2% | #30 of 48 | Epoch AI |