Llama 2-70B
Llama 2-70B ranks first on HELM Classic.
39
Benchmarks with a score
1
Ranked first, out of five or more models
3
In the top three
Meta Jul 2023 Llama 2-70B
Open weights Open model
113.7
#217 of 271
Epoch Capabilities Index
- GPQA Diamond 26.3% #208 /219
- Vectara Hallucination Leaderboard (HHEM) 5.90% #46 /148
- MMLU-CF 52.2% #29 /37
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TriviaQA | 87.6% | #5 of 39 | Model card | |
| BoolQ | 85% | #4 of 28 | Model card | |
| LAMBADA | 78.9% | #4 of 26 | Epoch AI | |
| Natural Questions | 39.5% | #5 of 20 | Model card | |
| Vectara Hallucination Leaderboard (HHEM) | 5.90% | #46 of 148 | Official leaderboard | |
| SQuAD v2 | 80.7% | #3 of 8 | Model card | |
| WinoGrande | 80.2% | #19 of 42 | Model card | |
| PIQA | 82.8% | #15 of 33 | Model card | |
| HellaSwag | 85.3% | #23 of 47 | Model card | |
| EQ-Bench v2 | 73.6 | #42 of 85 | Official leaderboard | |
| OpenBookQA | 60.2% | #10 of 20 | Model card | |
| AGIEval | 54.2% | #18 of 35 | Model card | |
| CommonSenseQA | 78.5% | #9 of 17 | Model card | |
| ARC-Easy | 80.2% | #16 of 23 | Model card | |
| TruthfulQA | 50.2% | #20 of 28 | Model card | |
| SIQA | 50.7% | #16 of 22 | Model card | |
| ARC-Challenge | 57.4% | #34 of 45 | Model card | |
| MMLU | 68.9% | #99 of 127 | Model card | |
| MMLU-CF | 52.2% | #29 of 37 | Official leaderboard | |
| MMLU-Pro | 37.5% | #170 of 212 | Official leaderboard | |
| AlpacaEval 2.0 | 14.7% | #33 of 40 | Official leaderboard | |
| BIG-Bench Hard | 51.2% | #62 of 71 | Model card | |
| LMArena (Elo) | 1,171 | #297 of 333 | Official leaderboard | |
| WildBench | 26.9 | #42 of 47 | Official leaderboard | |
| CommonsenseQA 2.0 | 50% | #6 of 6 | Epoch AI |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ForecastBench | 50.3 | #3 of 86 | Official leaderboard | |
| GSM8K | 56.8% | #79 of 87 | Model card | |
| MATH | 13.5% | #88 of 93 | Model card | |
| GPQA Diamond | 26.3% | #208 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 0% | #202 of 202 | Epoch AI | |
| DTBench | 2.76% | #178 of 178 | Official leaderboard | |
| MATH Level 5 | 3.29% | #94 of 94 | Epoch AI |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LegalBench | 62.4% | #127 of 138 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Situational Awareness Dataset (SAD) | 35.2% | #14 of 17 | Official leaderboard | |
| OR-Bench | 96.1% | #26 of 28 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Classic | 94.3 | #1 of 17 | Official leaderboard | |
| HELM Lite | 48.2 | #42 of 71 | Official leaderboard |
Meta's models over time
Meta's models in color, against every other model on the Epoch Capabilities Index, by release date.
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system cards
Every result in the longer documents the lab published with it. 2 of them aren't in the launch post.
Capabilities
- MMLU 68.9
- BIG-Bench Hard 51.2
- AGIEval 54.2
Safety
- TruthfulQA 50.2
- ToxiGen 24.6
Capabilities
- MMLU 68.9
- BIG-Bench Hard 51.2
- AGIEval 54.2
- HumanEval 29.9
- MBPP 45
- BoolQ 85
- PIQA 82.8
- SIQA 50.7
- HellaSwag 85.3
- WinoGrande 80.2
- ARC-Easy 80.2
- ARC-Challenge 57.4
- OpenBookQA 60.2
- CommonSenseQA 78.5
- Natural Questions 39.5
- TriviaQA 87.6
- SQuAD v2 80.7
- QuAC 42.4
- GSM8K 56.8
- MATH 13.5
- Llama 2-Chat helpfulness human evaluation (Meta)