- MMMU 55.1
- MMBench (dev-en) 86.7
- MMMU-Pro (std/vision) 38.5
- ScienceQA Visual (img-test) 97.5
- MathVista (testmini) 62.4
- InterGPS 48.6
- AI2D 82.3
- ChartQA 81.4
- DocVQA 93.2
- InfoVQA 72.7
- TextVQA (val) 75.6
- OCR Bench 84.4
- POPE 85.6
- BLINK 61.3
- Video MME 16 frames 55
Phi-4-multimodal
Phi-4-multimodal places in the top three on CoVoST2.
40
Benchmarks with a score
0
Ranked first, out of five or more models
1
In the top three
Microsoft Feb 2025 Phi-4-multimodal
Open weights Small
No Epoch Capabilities Index score yet
- MMMU-Pro 38.5% #69 /72
- OCRBench v2 38.1% #45 /65
- BLINK 61.3% #20 /28
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-CN v1.1 | 74.9% | #80 of 178 | Official leaderboard | |
| Longform Creative Writing (EQ-Bench) | 22.8 | #117 of 119 | Official leaderboard | |
| POPE | 85.6% | #7 of 7 | Model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HiPhO | 10.6% | #34 of 35 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SnakeBench | 20 | #127 of 183 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench | 83.4% | #30 of 155 | Official leaderboard | |
| A-Bench | 75.4% | #30 of 116 | Official leaderboard | |
| CoVoST2 | 39.3 | #3 of 11 | Model card | |
| CCBench (MMBench Chinese Culture) | 55.5% | #49 of 152 | Official leaderboard | |
| MMBench-EN v1.1 | 79.5% | #59 of 178 | Official leaderboard | |
| Common Voice | 6.80% | #1 of 3 | Model card | |
| FLEURS | 4.00% | #1 of 3 | Model card | |
| MMStar | 58.9% | #63 of 173 | Official leaderboard | |
| Inter-GPS (Geometry3K) | 48.6% | #1 of 2 | Model card | |
| ScienceQA | 97.5% | #1 of 2 | Model card | |
| DocVQA | 93.2% | #19 of 33 | Model card | |
| VoiceBench | 64.3% | #10 of 15 | Official leaderboard | |
| OCRBench v2 | 38.1% | #45 of 65 | Official leaderboard | |
| TextVQA | 75.6% | #12 of 17 | Model card | |
| BLINK | 61.3% | #20 of 28 | Model card | |
| InfographicVQA | 72.7% | #16 of 22 | Model card | |
| ChartQA | 81.4% | #27 of 37 | Model card | |
| MMAU v05.15.25 | 65.7% | #7 of 9 | Official leaderboard | |
| OCRBench | 84.4% | #22 of 28 | Model card | |
| Audio MC | 15.5% | #13 of 16 | Official leaderboard | |
| MathVista | 62.4% | #54 of 64 | Model card | |
| AI2D | 82.3% | #38 of 44 | Model card | |
| MMMU | 55.1% | #87 of 97 | Model card | |
| Video-MME | 55% | #40 of 42 | Model card | |
| MMMU-Pro | 38.5% | #69 of 72 | Model card | |
| MMAU | 55.6% | #14 of 14 | Model card | |
| AIR-Bench (Audio Instruction Benchmark) | 6.98 | #1 of 1 | Model card | |
| Golden3 speech summarization (Microsoft) | 6.28 | #1 of 1 | Model card | |
| Open ASR Leaderboard | 6.14% | #1 of 1 | Model card |
Safety and security
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| XSTest | 26.4% | #7 of 16 | Model card | |
| RTVLM | 6.39 | #1 of 2 | Model card | |
| Harmful content defect rate (Azure AI Evaluation SDK, Microsoft) | 4.00% | #3 of 3 | Model card | |
| Microsoft internal multimodal RAI benchmark | 7.96 | #2 of 2 | Model card | |
| VLGuard | 8.91 | #2 of 2 | Model card |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 64.7 | #58 of 173 | Official leaderboard |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 12 of them aren't in the launch post.
Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs 3 Mar 2025
Capabilities
- MMMU 55.1
- MMMU-Pro 38.5
- ScienceQA 97.5
- MathVista 62.4
- Inter-GPS (Geometry3K) 48.6
- MMBench 86.7
- AI2D 82.3
- ChartQA 81.4
- TextVQA 75.6
- DocVQA 93.2
- InfographicVQA 72.7
- OCRBench 84.4
- BLINK 61.3
- Video-MME 55
- Common Voice 6.8
- FLEURS 4
- Open ASR Leaderboard 6.14
- CoVoST2 39.3
- Golden3 speech summarization (Microsoft) 6.28
- AIR-Bench (Audio Instruction Benchmark) 6.98
- MMAU 55.6
Hallucination
- POPE 85.6