Mistral Small 3.1 places in the top three on GPQA Main.
Mistral AI Released 17 Mar 2025 Mistral AI Mar 2025 Open weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (HHEM) | 5.60% | #47 of 148 | Official leaderboard | |
| MMLU | 80.6% | #71 of 127 | Model card | |
| LMArena (Elo) | 1,303 | #229 of 333 | Official leaderboard | |
| MMLU-Pro | 66.8% | #119 of 151 | Model card | |
| Global MMLU Lite | 78.5% | #34 of 43 | Official leaderboard | |
| SimpleQA | 10.4% | #48 of 59 | Model card | |
| RULER | 81.2% | #31 of 38 | Model card | |
| FACTS Multimodal | 21.2% | #36 of 39 | Official leaderboard | |
| FACTS Parametric | 5.47% | #42 of 45 | Official leaderboard | |
| EQ-Bench 3 | 832 | #73 of 78 | Official leaderboard | |
| Creative Writing v3 | 42.3 | #118 of 120 | Official leaderboard | |
| LongBench v2 | 37.2% | #27 of 27 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GPQA Main | 44.4% | #2 of 7 | Model card | |
| MATH | 69.3% | #43 of 93 | Model card | |
| MATH Level 5 | 46.8% | #54 of 94 | Epoch AI | |
| GPQA Diamond | 47.5% | #156 of 219 | Epoch AI | |
| DTBench | 58.6% | #139 of 177 | Epoch AI | |
| CritPt | 0% | #128 of 160 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 5.83% | #170 of 202 | Epoch AI | |
| Misguided Attention | 26% | #41 of 45 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Galileo Agent Leaderboard | 0.91 | #6 of 31 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 57.7% | #31 of 67 | Official leaderboard | |
| ITBench | 32.9% | #12 of 13 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MortgageTax | 61.2% | #58 of 86 | Official leaderboard | |
| Vals Finance Agent v1 | 10% | #40 of 47 | Official leaderboard | |
| LegalBench | 69.2% | #118 of 138 | Official leaderboard | |
| FailureSensorIQ | 47.9% | #21 of 24 | Official leaderboard | |
| CorpFin v2 | 44.2% | #106 of 119 | Official leaderboard | |
| TaxEval v2 | 58.3% | #119 of 129 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AI2D | 93.7% | #10 of 44 | Model card | |
| DocVQA | 94.1% | #16 of 33 | Model card | |
| ChartQA | 86.2% | #20 of 37 | Model card | |
| MathVista | 68.9% | #44 of 64 | Model card | |
| MMMU | 64% | #74 of 97 | Model card | |
| Vision Arena | 1,127 | #109 of 136 | Official leaderboard | |
| MM-MT-Bench | 7.30 | #5 of 6 | Model card | |
| MMMU-Pro | 49.3% | #66 of 72 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BaxBench | 10.8% | #25 of 34 | Official leaderboard |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LiveBench | 44% | #33 of 48 | Epoch AI | |
| Political Science LLM Benchmark | 64.2% | #22 of 29 | Official leaderboard | |
| HELM Capabilities | 55.8 | #49 of 60 | Official leaderboard |
Mistral AI's models in color, against every other model on the Epoch Capabilities Index, by release date.
The benchmarks the lab chose to headline, with the scores it gave.