- AlpacaEval 2 LC 9.1
- BBH 35
- DROP 34.6
- GSM8K 68.3
- IFEval 70.1
- MATH 20.7
- MMLU 40
- Safety (Ai2 safety suite average) 87.6
- PopQA 12.9
- TruthQA 48.7
OLMo 2 1B Instruct
OLMo 2 1B Instruct ranks first on Tulu 3 Safety Eval.
10
Benchmarks with a score
1
Ranked first, out of five or more models
1
In the top three
Ai2 Apr 2025 OLMo 2 1B Instruct
Open weights Small
No Epoch Capabilities Index score yet
- PopQA 12.9% #7 /7
- IFEval 70.1% #117 /126
- MMLU 40% #126 /127
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TruthfulQA | 48.7% | #21 of 28 | Model card | |
| IFEval | 70.1% | #117 of 126 | Model card | |
| BIG-Bench Hard | 35% | #68 of 71 | Model card | |
| MMLU | 40% | #126 of 127 | Model card | |
| DROP | 34.6% | #40 of 40 | Model card | |
| AlpacaEval 2.0 | 9.10% | #16 of 16 | Model card | |
| PopQA | 12.9% | #7 of 7 | Model card |
Math and science
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Tulu 3 Safety Eval | 87.6% | #1 of 6 | Model card |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it.
2 OLMo 2 Furious 8 Oct 2025
Capabilities
- AlpacaEval 2.0 9.1
- BIG-Bench Hard 35
- DROP 34.6
- GSM8K 68.3
- IFEval 70.1
- MATH 20.7
- MMLU 40
- PopQA 12.9
Hallucination
- TruthfulQA 48.7
Safety
- Tulu 3 Safety Eval 87.6