- Arena-Hard (thinking=True) 57.6
- AlpacaEval-2.0 (thinking=True) 62.7
- MMLU 65.5
- PopQA 26.2
- TruthfulQA 66.9
- BigBenchHard 69.1
- DROP 59.4
- GSM8K 80.9
- HumanEval 89.7
- HumanEval+ 86.1
- IFEval 74.8
- AttaQ 88.5
- AIME24 8.12
- MATH-500 69
Granite 3.3 8B Instruct
Granite 3.3 8B Instruct ranks first on AttaQ.
20
Benchmarks with a score
1
Ranked first, out of five or more models
1
In the top three
IBM Apr 2025
Granite 3.3 8B Instruct
Open weights Flagship
No Epoch Capabilities Index score yet
- PopQA 26.2% #6 /7
- IFEval 74.8% #111 /126
- MMLU 65.5% #106 /127
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TruthfulQA | 66.9% | #8 of 28 | Model card | |
| AlpacaEval 2.0 | 62.7% | #5 of 16 | Model card | |
| Vectara Hallucination Leaderboard (2025 dataset) | 10.6% | #66 of 104 | Official leaderboard | |
| BIG-Bench Hard | 69.1% | #47 of 71 | Model card | |
| Arena-Hard-Auto | 57.6% | #31 of 39 | Model card | |
| MMLU | 65.5% | #106 of 127 | Model card | |
| PopQA | 26.2% | #6 of 7 | Model card | |
| IFEval | 74.8% | #111 of 126 | Model card | |
| DROP | 59.4% | #37 of 40 | Model card |
Math and science
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval | 89.7% | #14 of 95 | Model card | |
| HumanEval+ | 86.1% | #13 of 27 | Model card |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AttaQ | 88.5% | #1 of 5 | Model card | |
| AIR-Bench 2024 | 82.5% | #19 of 82 | Official leaderboard | |
| Anthropic Red Team | 99.1% | #33 of 75 | Official leaderboard | |
| HELM Safety | 90.5 | #45 of 75 | Official leaderboard | |
| SimpleSafetyTests | 97.5% | #51 of 75 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Capabilities | 46.3 | #57 of 60 | Official leaderboard |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.