Mistral Medium
Mistral Medium places in the top three on AlpacaEval.
Mistral AI Released 11 Dec 2023 9
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Mistral AI Dec 2023 Mistral Medium
Closed weights
No Epoch Capabilities Index score yet
- PRBench 39.5% #34 /44
- Phare 58.1% #57 /62
- MixEval 81.9% #14 /34
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| AlpacaEval | 96.8% | #2 of 19 | Official leaderboard | |
| EQ-Bench v2 | 82.6 | #13 of 85 | Official leaderboard | |
| MixEval | 81.9% | #14 of 34 | Official leaderboard | |
| AlpacaEval 2.0 | 28.6% | #17 of 40 | Official leaderboard | |
| LMArena (Elo) | 1,223 | #274 of 333 | Official leaderboard | |
| Phare | 58.1% | #57 of 62 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DTBench | 53.3% | #150 of 177 | Epoch AI |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PRBench | 39.5% | #34 of 44 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OR-Bench | 14% | #8 of 28 | Official leaderboard |