Llama 2-34B.
Meta Released 18 Jul 2023 Meta Jul 2023 Closed weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TriviaQA | 84.5% | #11 of 39 | Model card | |
| BoolQ | 83.7% | #9 of 28 | Model card | |
| Natural Questions | 32.8% | #9 of 20 | Model card | |
| WinoGrande | 76.7% | #21 of 42 | Model card | |
| PIQA | 81.9% | #17 of 33 | Model card | |
| HellaSwag | 83.3% | #27 of 47 | Model card | |
| SQuAD v2 | 77.4% | #5 of 8 | Model card | |
| CommonSenseQA | 74.3% | #11 of 17 | Model card | |
| OpenBookQA | 58.2% | #13 of 20 | Model card | |
| SIQA | 50.9% | #15 of 22 | Model card | |
| AGIEval | 43.4% | #27 of 35 | Model card | |
| ARC-Easy | 79.4% | #18 of 23 | Model card | |
| ARC-Challenge | 54.5% | #37 of 45 | Model card | |
| TruthfulQA | 43.5% | #24 of 28 | Model card | |
| MMLU | 62.6% | #111 of 127 | Model card | |
| BIG-Bench Hard | 44.1% | #64 of 71 | Model card |
Meta's models in color, against every other model on the Epoch Capabilities Index, by release date.
Every result in the longer documents the lab published with it. 25 of them aren't in the launch post.