DeepSeek-Coder-V2 236B
DeepSeek-Coder-V2 236B ranks first on GSM8K.
DeepSeek Released 17 Jun 2024 15
Benchmarks with a score
1
Ranked first, out of five or more models
1
In the top three
DeepSeek Jun 2024 DeepSeek-Coder-V2 236B
Open weights
No Epoch Capabilities Index score yet
- BigCodeBench-Hard 29.4% #22 /96
- AutoCodeBench 37.7% #23 /34
- OlympicArena 34.1% #4 /25
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| WinoGrande | 83.7% | #8 of 76 | Epoch AI | |
| WildBench | 48.9 | #13 of 47 | Official leaderboard | |
| ARC-Challenge | 64.3% | #30 of 75 | Epoch AI | |
| MMLU-Pro | 63.6% | #105 of 212 | Official leaderboard | |
| LMArena (Elo) | 1,265 | #254 of 333 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 94.5% | #1 of 82 | Epoch AI | |
| OlympicArena | 34.1% | #4 of 25 | Official leaderboard | |
| Omni-MATH | 25.8% | #6 of 9 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench | 54% | #4 of 69 | Official leaderboard | |
| MBPP+ | 75.1% | #4 of 35 | Official leaderboard | |
| HumanEval+ | 82.3% | #9 of 46 | Official leaderboard | |
| BigCodeBench-Hard | 29.4% | #22 of 96 | Official leaderboard | |
| SciCode | 3.10% | #6 of 18 | Official leaderboard | |
| AutoCodeBench | 37.7% | #23 of 34 | Official leaderboard | |
| BigO(Bench) | 29.6% | #15 of 18 | Official leaderboard |