phi-3-small 7.4B ranks first on Microsoft RAI benchmark: Harmful Content Continuation (DR-3).
Microsoft Released 23 Apr 2024 Microsoft Apr 2024 Open weights
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| PIQA | 86.9% | #3 of 33 | Model card | |
| SIQA | 79.2% | #2 of 22 | Model card | |
| ARC-Easy | 97% | #4 of 23 | Model card | |
| TruthfulQA | 70.2% | #5 of 28 | Model card | |
| ANLI | 58.1% | #2 of 11 | Epoch AI | |
| OpenBookQA | 88% | #4 of 20 | Model card | |
| BoolQ | 84.8% | #6 of 28 | Model card | |
| WinoGrande | 81.5% | #14 of 42 | Model card | |
| ARC-Challenge | 90.7% | #16 of 45 | Model card | |
| CommonSenseQA | 80% | #8 of 17 | Model card | |
| EQ-Bench v2 | 73.5 | #44 of 85 | Official leaderboard | |
| BIG-Bench Hard | 79.1% | #37 of 71 | Model card | |
| MT-Bench | 8.70 | #13 of 23 | Model card | |
| MMLU | 75.7% | #89 of 127 | Model card | |
| AGIEval | 45.1% | #26 of 35 | Model card | |
| Microsoft RAI benchmark: Ungroundedness | 0.30 | #3 of 4 | Model card | |
| HellaSwag | 77% | #39 of 47 | Model card | |
| LMArena (Elo) | 1,171 | #295 of 333 | Official leaderboard | |
| TriviaQA | 58.1% | #35 of 39 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| GSM8K | 89.6% | #39 of 87 | Model card | |
| MATH | 34.6% | #77 of 93 | Model card | |
| GPQA Diamond | 34.3% | #268 of 284 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedQA | 65.4% | #3 of 4 | Model card |
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Microsoft RAI benchmark: Harmful Content Continuation (DR-3) | 0.30% | #1 of 5 | Model card | |
| Microsoft RAI benchmark: Harmful Content Summarization (DR-3) | 11% | #3 of 5 | Model card | |
| Microsoft RAI benchmark: Jailbreak (DR-1) | 10.7% | #3 of 5 | Model card | |
| Microsoft RAI benchmark: Third Party Harm (DR-1) | 25.3% | #5 of 5 | Model card |
Microsoft's models in color, against every other model on the Epoch Capabilities Index, by release date.
Every result in the longer documents the lab published with it. 27 of them aren't in the launch post.