Mistral Large 2 Nov 2024
Mistral Large 2 places in the top three on LLM-AggreFact.
Mistral AI Released 15 Nov 2024 35
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Mistral AI Nov 2024 Mistral Large 2
Nov 2024 snapshot Open weights
128.6
#170 of 271
Epoch Capabilities Index
- SimpleQA Verified 22.3% #38 /56
- GPQA Diamond 51.3% #143 /219
- FrontierMath 0.34% #67 /70
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (2025 dataset) | 4.50% | #8 of 104 | Official leaderboard | |
| LLM-AggreFact | 76.5% | #2 of 6 | Official leaderboard | |
| MMLU-Pro | 67.9% | #92 of 212 | Official leaderboard | |
| BuzzBench | 47.5 | #12 of 23 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.10% | #84 of 148 | Official leaderboard | |
| SuperGPQA | 40.6% | #35 of 58 | Official leaderboard | |
| ECLeKTic | 13.4% | #22 of 35 | Official leaderboard | |
| LMArena (Elo) | 1,306 | #224 of 333 | Official leaderboard | |
| SimpleQA Verified | 22.3% | #38 of 56 | Official leaderboard | |
| LongBench v2 | 39.6% | #23 of 33 | Official leaderboard | |
| FACTS Grounding | 72.6% | #24 of 34 | Official leaderboard | |
| LLM Confabulation Leaderboard | 21.4% | #41 of 58 | Official leaderboard | |
| WritingBench | 54.4 | #35 of 45 | Official leaderboard | |
| EQ-Bench 3 | 998 | #63 of 78 | Official leaderboard | |
| HELM Arabic | 71.7 | #23 of 28 | Official leaderboard | |
| HELM Arabic Enterprise | 62.0 | #19 of 23 | Official leaderboard | |
| Creative Writing v3 | 51.9 | #101 of 120 | Official leaderboard | |
| Phare | 58.8% | #54 of 62 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ForecastBench | 61.5 | #24 of 83 | Epoch AI | |
| MATH Level 5 | 50.3% | #51 of 94 | Epoch AI | |
| U-MATH | 47.6% | #17 of 30 | Official leaderboard | |
| GPQA Diamond | 51.3% | #143 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 7.78% | #157 of 202 | Epoch AI | |
| DTBench | 34.7% | #140 of 178 | Official leaderboard | |
| FrontierMath | 0.34% | #67 of 70 | Epoch AI |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SWE-PolyBench | 5.88% | #5 of 6 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BFCL V4 | 38.4% | #29 of 65 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 58% | #30 of 67 | Official leaderboard | |
| Elimination Game | 4 | #28 of 56 | Official leaderboard | |
| Galileo Agent Leaderboard | 0.81 | #23 of 31 | Official leaderboard | |
| LisanBench | 138.0 | #102 of 133 | Official leaderboard | |
| Step Race | 1 | #48 of 59 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| TaxEval v2 | 63.8% | #107 of 129 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LiveBench | 48.4% | #28 of 48 | Epoch AI | |
| HELM Capabilities | 59.8 | #43 of 60 | Official leaderboard |
Mistral AI's models over time
Mistral AI's models in color, against every other model on the Epoch Capabilities Index, by release date.