- MMLU (CoT), 0-shot, macro_avg/acc 86
- MMLU Pro (CoT), 5-shot, macro_avg/acc 68.9
- IFEval 92.1
- GPQA Diamond (CoT), 0-shot 50.5
- HumanEval, 0-shot, pass@1 88.4
- MBPP EvalPlus (base), 0-shot, pass@1 87.6
- MATH (CoT), 0-shot, sympy_intersection_score 77
- BFCL v2, overall_ast_summary/macro_avg/valid 77.3
- MGSM, 0-shot, em 91.1
Llama 3.3 70B Instruct
Llama 3.3 70B Instruct places in the top three on 2 benchmarks.
47
Benchmarks with a score
0
Ranked first, out of five or more models
2
In the top three
Meta Dec 2024 Llama 3.3 70B Instruct
Open weights Open model
No Epoch Capabilities Index score yet
- Meta Agents Research Environments (ARE) / Gaia2 4.40% #13 /13
- LiveSQLBench 12.2% #33 /36
- Vectara Hallucination Leaderboard (HHEM) 4.00% #86 /148
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (2025 dataset) | 4.10% | #4 of 104 | Official leaderboard | |
| ArabicMMLU | 69.8% | #3 of 39 | Official leaderboard | |
| ArbMMLU-HT | 67.8% | #3 of 39 | Official leaderboard | |
| Open Arabic LLM Leaderboard v2 | 69.3% | #5 of 39 | Official leaderboard | |
| IFEval | 92.1% | #17 of 126 | Model card | |
| MGSM | 91.1% | #8 of 52 | Model card | |
| MMLU-CF | 68.8% | #6 of 37 | Official leaderboard | |
| MadinahQA | 53.1% | #10 of 39 | Official leaderboard | |
| MMLU | 86% | #49 of 127 | Model card | |
| FaithJudge | 16.4% | #15 of 37 | Official leaderboard | |
| AlGhafa | 63.7% | #17 of 39 | Official leaderboard | |
| ALRAGE | 75.8% | #17 of 39 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.00% | #86 of 148 | Official leaderboard | |
| LMArena (Elo) | 1,318 | #209 of 333 | Official leaderboard | |
| SuperGPQA | 37.7% | #38 of 58 | Official leaderboard | |
| Wer wird Millionär? LLM Benchmark | 353.0 | #52 of 76 | Official leaderboard | |
| BuzzBench | 45.7 | #16 of 23 | Official leaderboard | |
| HELM Arabic | 76.4 | #20 of 28 | Official leaderboard | |
| Phare | 62% | #47 of 62 | Official leaderboard | |
| MMLU-Pro | 68.9% | #116 of 151 | Model card | |
| SEA-HELM | 60.4 | #49 of 62 | Official leaderboard | |
| LLM-AggreFact | 74.5% | #5 of 6 | Official leaderboard | |
| HELM Arabic Enterprise | 61.5 | #20 of 23 | Official leaderboard | |
| WritingBench | 48.0 | #41 of 45 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH | 77% | #28 of 93 | Model card | |
| ForecastBench | 61.1 | #27 of 86 | Official leaderboard | |
| U-MATH | 44.7% | #18 of 30 | Official leaderboard | |
| Misguided Attention | 35.3% | #30 of 45 | Official leaderboard | |
| GPQA Diamond | 50.5% | #230 of 284 | Model card | |
| SimpleBench | 19.9% | #93 of 100 | Official leaderboard | |
| PolyMath | 11.5% | #38 of 38 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MBPP | 87.6% | #5 of 66 | Model card | |
| BigCodeBench | 52.2% | #12 of 69 | Official leaderboard | |
| HumanEval | 88.4% | #21 of 95 | Model card | |
| BigCodeBench-Hard | 28.4% | #27 of 96 | Official leaderboard | |
| Aider-Edit | 59.4% | #23 of 37 | Official leaderboard | |
| LiveSQLBench | 12.2% | #33 of 36 | Official leaderboard | |
| WeirdML | 14.4% | #106 of 112 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Berkeley Function Calling Leaderboard (BFCL) | 77.3% | #5 of 47 | Model card | |
| BFCL V4 | 31.9% | #37 of 65 | Official leaderboard | |
| SnakeBench | 21 | #109 of 183 | Official leaderboard | |
| Galileo Agent Leaderboard v2 | 20% | #19 of 21 | Official leaderboard | |
| Meta Agents Research Environments (ARE) / Gaia2 | 4.40% | #13 of 13 | Official leaderboard |
Professional work
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AraTrust | 88.3% | #5 of 39 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HELM Lite | 81.2 | #13 of 71 | Official leaderboard |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it.
Capabilities
- MMLU 86
- MMLU-Pro 68.9
- IFEval 92.1
- GPQA Diamond 50.5
- HumanEval 88.4
- MBPP 87.6
- MATH 77
- Berkeley Function Calling Leaderboard (BFCL) 77.3
- MGSM 91.1