- MMMU 77.9
- MMMU-Pro 67.6
- MathVision 68.7
- OlympiadBench 65
- MathVista 85.6
- V* 89
- VLM are Blind 92.1
- ZeroBench (main) 2 (sub 30.8) 2
- VisuLogic 35
- RealWorldQA 78.4
- SimpleVQA 63.4
- MMStar 77.8
- MMBench-en (cn 89.1) 89.9
- MMVP 69.3
- HallusionBench 60.3
- TextVQA 81.8
- AI2D 87.3
- ChartQA 89.1
- InfographicVQA 91.2
- DocVQA 96.9
- CharXiv (RQ) 60.2 (DQ 92.6) 60.2
- BLINK 72.1
- VisualWebBench 87.3
- RefCOCO-avg 91.3
- CountBench 93.7
- All-Angles Bench 58.6
- MotionBench 68.4
- MVBench 74.4
- TOMATO 44.7
- TVBench 63.6
- TempCompass 83.7
- LongVideoBench 74
- LVBench 64.6
- MLVU 82.1
- VideoMME (w/o sub) 77.9
- OVBench 60
- OVOBench 72.3
- StreamingBench (proactive), thinking 68.0 (non-thinking 82.8) 68
- Video-MMMU 81.4
- MMVU 70.1
- Charades-STA (mIoU) 64
- ScreenSpot-V2 95.2
- ScreenSpot-Pro 60.9
- OSWorld 36.7
- Windows Agent Arena 39.6
- WebVoyager 87.2
- Online-Mind2Web 76.4
- Android World 62.1
- OCRBench 861
- LVIS-MG 72.5
- Dream-1K 43.9
- TemporalBench 79.8
- StreamBench 72.8
- TACoS (mIoU) 49.6
Seed1.5-VL
Seed1.5-VL ranks first on 2 of the 48 benchmarks it has scores on.
58
Benchmarks with a score
2
Ranked first, out of five or more models
12
In the top three
ByteDance Seed May 2025 Seed1.5-VL
Closed weights Flagship
No Epoch Capabilities Index score yet
- MMMU-Pro 67.6% #46 /72
- SimpleVQA 63.4% #15 /23
- MMVU 70.1% #13 /20
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| SimpleVQA | 63.4% | #15 of 23 | Model card | |
| HallusionBench | 60.3% | #19 of 23 | Model card |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OlympiadBench | 65% | #3 of 3 | Model card |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ScreenSpot-v2 | 95.2% | #1 of 14 | Model card | |
| WebVoyager | 87.2% | #3 of 11 | Model card | |
| Online-Mind2Web | 76.4% | #3 of 8 | Model card | |
| VisualWebBench | 87.3% | #1 of 2 | Model card | |
| Windows Agent Arena | 39.6% | #3 of 5 | Model card | |
| OSWorld | 36.7% | #8 of 12 | Model card | |
| OSWorld-Verified | 33.8% | #25 of 33 | Official leaderboard | |
| ScreenSpot-Pro | 60.9% | #19 of 25 | Model card | |
| AndroidWorld | 62.1% | #8 of 10 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| DocVQA | 96.9% | #1 of 33 | Model card | |
| MMMU | 77.9% | #13 of 136 | Official leaderboard | |
| InfographicVQA | 91.2% | #3 of 22 | Model card | |
| BLINK | 72.1% | #5 of 28 | Model card | |
| MathVista | 85.6% | #12 of 64 | Model card | |
| MMBench | 89.9% | #2 of 9 | Model card | |
| ChartQA | 89.1% | #9 of 37 | Model card | |
| All-Angles Bench | 58.6% | #2 of 6 | Model card | |
| Charades-STA | 64% | #1 of 3 | Model card | |
| MMStar | 77.8% | #11 of 30 | Model card | |
| TextVQA | 81.8% | #7 of 17 | Model card | |
| RefCOCO / RefCOCO+ / RefCOCOg | 91.3% | #8 of 19 | Model card | |
| CharXiv (DQ) | 92.6% | #3 of 7 | Model card | |
| TempCompass | 83.7% | #3 of 7 | Model card | |
| VisuLogic | 35% | #3 of 7 | Model card | |
| CountBench | 93.7% | #9 of 20 | Model card | |
| RealWorldQA | 78.4% | #15 of 30 | Model card | |
| V* | 89% | #7 of 12 | Model card | |
| MMVP | 69.3% | #3 of 5 | Model card | |
| OCRBench | 86.1% | #17 of 28 | Model card | |
| MATH-Vision (MathVision) | 68.7% | #26 of 42 | Model card | |
| VLMs Are Blind | 92.1% | #5 of 8 | Model card | |
| Video-MMMU | 81.4% | #19 of 30 | Model card | |
| AI2D | 87.3% | #28 of 44 | Model card | |
| LongVideoBench | 74% | #7 of 11 | Model card | |
| MMMU-Pro | 67.6% | #46 of 72 | Model card | |
| MMVU | 70.1% | #13 of 20 | Model card | |
| MotionBench | 68.4% | #6 of 9 | Model card | |
| MLVU | 82.1% | #11 of 16 | Model card | |
| Video-MME | 77.9% | #29 of 42 | Model card | |
| LVBench | 64.6% | #18 of 25 | Model card | |
| MVBench | 74.4% | #11 of 15 | Model card | |
| CharXiv | 60.2% | #57 of 67 | Model card | |
| ZeroBench | 2.00% | #23 of 25 | Model card | |
| TVBench | 63.6% | #7 of 7 | Model card | |
| OVBench | 60% | #5 of 5 | Model card | |
| OVO-Bench | 72.3% | #5 of 5 | Model card | |
| SUNRGBD | 32.0 | #5 of 5 | Model card | |
| TOMATO | 44.7% | #5 of 5 | Model card | |
| StreamingBench | 68% | #3 of 3 | Model card | |
| DREAM-1K | 43.9% | #1 of 1 | Model card | |
| LVIS-MG | 72.5% | #1 of 1 | Model card | |
| Seed1.5-VL internal benchmark (ByteDance Seed in-house) | 59.3% | #1 of 1 | Model card | |
| StreamBench (StreamChat) | 72.8% | #1 of 1 | Model card | |
| TACoS | 49.6% | #1 of 1 | Model card | |
| TemporalBench | 79.8% | #1 of 1 | Model card |
Its launch post
The benchmarks the lab chose to headline, with the scores it gave.
Its system card
Every result in the longer documents the lab published with it. 3 of them aren't in the launch post.
Seed1.5-VL Technical Report 11 May 2025
Capabilities
- MMMU 77.9
- MMMU-Pro 67.6
- MATH-Vision (MathVision) 68.7
- OlympiadBench 65
- MathVista 85.6
- V* 89
- VLMs Are Blind 92.1
- VisuLogic 35
- RealWorldQA 78.4
- SimpleVQA 63.4
- MMStar 77.8
- MMVP 69.3
- HallusionBench 60.3
- TextVQA 81.8
- AI2D 87.3
- ChartQA 89.1
- InfographicVQA 91.2
- DocVQA 96.9
- CharXiv 60.2
- CharXiv (DQ) 92.6
- BLINK 72.1
- LVIS-MG 72.5
- VisualWebBench 87.3
- RefCOCO / RefCOCO+ / RefCOCOg 91.3
- CountBench 93.7
- All-Angles Bench 58.6
- ZeroBench 2
- MMBench 89.9
- OCRBench 86.1
- SUNRGBD 32
- MotionBench 68.4
- MVBench 74.4
- TOMATO 44.7
- TVBench 63.6
- DREAM-1K 43.9
- TempCompass 83.7
- LongVideoBench 74
- LVBench 64.6
- MLVU 82.1
- Video-MME 77.9
- TemporalBench 79.8
- OVBench 60
- OVO-Bench 72.3
- StreamBench (StreamChat) 72.8
- StreamingBench 68
- Video-MMMU 81.4
- MMVU 70.1
- Charades-STA 64
- TACoS 49.6
- ScreenSpot-v2 95.2
- ScreenSpot-Pro 60.9
- OSWorld 36.7
- Windows Agent Arena 39.6
- WebVoyager 87.2
- Online-Mind2Web 76.4
- AndroidWorld 62.1
- Seed1.5-VL internal benchmark (ByteDance Seed in-house) 59.3