- MMLU (simple-evals) 84.8
- GPQA (simple-evals) 56.1
- MATH (simple-evals) 80.4
- HumanEval (simple-evals) 82.6
- MGSM (simple-evals) 80.6
- SimpleQA (simple-evals) 3
- DROP (simple-evals) 75.5
- MMLUPro 70.4
- ArenaHard 75.4
- LiveBench 47.6
- IFEval 63
- PhiBench (internal) 56.2
Phi-4
Phi-4 ranks first on Microsoft RAI benchmark: Jailbreak (DR-1).
50
Benchmarks with a score
1
Ranked first, out of five or more models
4
In the top three
Microsoft Dec 2024 Phi-4
Open weights Small
130.6
#158 of 271
Epoch Capabilities Index
- CritPt 0% #121 /160
- GPQA Diamond 56.1% #134 /219
- Vectara Hallucination Leaderboard (HHEM) 4.70% #64 /148
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Vectara Hallucination Leaderboard (2025 dataset) | 3.70% | #3 of 104 | Official leaderboard | |
| MMLU-CF | 67.8% | #8 of 37 | Official leaderboard | |
| ALRAGE | 77.2% | #14 of 39 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.70% | #64 of 148 | Official leaderboard | |
| FaithJudge | 17% | #16 of 37 | Official leaderboard | |
| MMLU | 84.8% | #56 of 127 | Model card | |
| ArbMMLU-HT | 39.1% | #21 of 39 | Official leaderboard | |
| MGSM | 80.6% | #29 of 52 | Model card | |
| Arena-Hard-Auto | 75.4% | #22 of 39 | Model card | |
| MadinahQA | 39.2% | #24 of 39 | Official leaderboard | |
| ArabicMMLU | 39.8% | #25 of 39 | Official leaderboard | |
| SuperGPQA | 37.6% | #39 of 58 | Official leaderboard | |
| Wer wird Millionär? LLM Benchmark | 349.0 | #53 of 76 | Official leaderboard | |
| Open Arabic LLM Leaderboard v2 | 45.7% | #28 of 39 | Official leaderboard | |
| MMLU-Pro | 70.4% | #111 of 151 | Model card | |
| DROP | 75.5% | #31 of 40 | Model card | |
| LMArena (Elo) | 1,256 | #259 of 333 | Official leaderboard | |
| BuzzBench | 38.7 | #18 of 23 | Official leaderboard | |
| AlGhafa | 42.3% | #32 of 39 | Official leaderboard | |
| LLM Confabulation Leaderboard | 29.4% | #53 of 58 | Official leaderboard | |
| Lech Mazur Creative Writing | 6.26 | #41 of 44 | Epoch AI | |
| IFEval | 63% | #121 of 126 | Model card | |
| SimpleQA | 3.00% | #58 of 59 | Model card | |
| Microsoft RAI benchmark: Grounding | 4.62 | #1 of 1 | Model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MATH | 80.4% | #21 of 93 | Model card | |
| MATH Level 5 | 64.9% | #38 of 94 | Epoch AI | |
| U-MATH | 54.5% | #13 of 30 | Official leaderboard | |
| GPQA Diamond | 56.1% | #134 of 219 | Epoch AI | |
| OTIS Mock AIME 2024-2025 | 13.8% | #150 of 202 | Epoch AI | |
| CritPt | 0% | #121 of 160 | Epoch AI | |
| PolyMath | 17.4% | #35 of 38 | Official leaderboard | |
| AMC 10/12 (November 2024) | 91.8 | #1 of 1 | Model card |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BigCodeBench-Hard | 27.4% | #30 of 96 | Official leaderboard | |
| BigCodeBench | 50.5% | #22 of 69 | Official leaderboard | |
| HumanEval | 82.6% | #47 of 95 | Model card | |
| HumanEval+ | 82.8% | #17 of 27 | Model card | |
| BIRD-CRITIC | 22% | #6 of 9 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BFCL V4 | 28.8% | #41 of 65 | Official leaderboard | |
| Berkeley Function Calling Leaderboard (BFCL) | 47% | #55 of 67 | Official leaderboard | |
| SnakeBench | 18 | #151 of 183 | Official leaderboard | |
| Elimination Game | 3 | #48 of 56 | Official leaderboard | |
| BALROG | 11.6% | #31 of 35 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| LEXam | 38.5% | #24 of 31 | Official leaderboard |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| Microsoft RAI benchmark: Jailbreak (DR-1) | 7.30% | #1 of 5 | Model card | |
| Microsoft RAI benchmark: Harmful Content Summarization (DR-3) | 10.2% | #2 of 5 | Model card | |
| Microsoft RAI benchmark: Third Party Harm (DR-1) | 12.1% | #2 of 5 | Model card | |
| AraTrust | 47.6% | #30 of 39 | Official leaderboard | |
| Microsoft RAI benchmark: Harmful Content Continuation (DR-3) | 3.60% | #5 of 5 | Model card |
Microsoft's models over time
Microsoft's models in color, against every other model on the Epoch Capabilities Index, by release date.
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 8 of them aren't in the launch post.
Phi-4 Technical Report 12 Dec 2024
Capabilities
- MMLU 84.8
- GPQA Diamond 56.1
- MATH 80.4
- HumanEval 82.6
- MGSM 80.6
- DROP 75.5
- MMLU-Pro 70.4
- HumanEval+ 82.8
- Arena-Hard-Auto 75.4
- LiveBench 47.6
- IFEval 63
- PhiBench 56.2
- AMC 10/12 (November 2024) 91.8
- HELMET