- ArenaHard 55.3
- Alpaca-Eval-2 61.2
- MMLU 66.8
- PopQA 28
- TruthfulQA 66.9
- BigBenchHard 64.8
- DROP 51
- GSM8K 81.7
- HumanEval 89.3
- HumanEval+ 85.7
- IFEval 74.3
- AttaQ 85.4
Granite 3.2 8B Instruct
Granite 3.2 8B Instruct.
14
Benchmarks with a score
0
Ranked first, out of five or more models
0
In the top three
IBM Feb 2025
Granite 3.2 8B Instruct
Open weights Flagship
No Epoch Capabilities Index score yet
- PopQA 28% #5 /7
- Vectara Hallucination Leaderboard (HHEM) 8.70% #24 /148
- IFEval 74.3% #113 /126
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) | 8.70% | #24 of 148 | Official leaderboard | |
| TruthfulQA | 66.9% | #7 of 28 | Model card | |
| AlpacaEval 2.0 | 61.2% | #6 of 16 | Model card | |
| PopQA | 28% | #5 of 7 | Model card | |
| BIG-Bench Hard | 64.8% | #53 of 71 | Model card | |
| MMLU | 66.8% | #103 of 127 | Model card | |
| Arena-Hard-Auto | 55.3% | #32 of 39 | Model card | |
| IFEval | 74.3% | #113 of 126 | Model card | |
| DROP | 51% | #38 of 40 | Model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 81.7% | #57 of 87 | Model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 89.3% | #15 of 95 | Model card | |
| HumanEval+ | 85.7% | #14 of 27 | Model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BFCL V4 | 26.9% | #49 of 65 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AttaQ | 85.4% | #4 of 5 | Model card |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.