Qwen2-VL-72B-Instruct
Qwen2-VL-72B-Instruct ranks first on TOMATO.
Released 29 Aug 2024
29
Benchmarks with a score
1
Ranked first, out of five or more models
4
In the top three
Unknown lab Aug 2024
Qwen2-VL-72B-Instruct
No Epoch Capabilities Index score yet
- ZeroBench 1.00% #59 /66
- VisualPuzzles 34.9% #15 /29
- HumanEval-V 14.2% #9 /24
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-CN v1.1 | 85.8% | #11 of 178 | Official leaderboard | |
| Creation-MMBench | -22.45 | #11 of 19 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| U-MATH | 41.8% | #22 of 30 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval-V | 14.2% | #9 of 24 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| BALROG | 12.8% | #32 of 37 | Epoch AI | |
| ScreenSpot-Pro | 1.01% | #11 of 11 | Official leaderboard |
Professional work
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MedXpertQA-MM | 28.4% | #8 of 9 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| A-Bench | 79.4% | #3 of 116 | Official leaderboard | |
| TOMATO | 37.9% | #1 of 22 | Official leaderboard | |
| MMBench | 86.9% | #8 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 86.1% | #12 of 178 | Official leaderboard | |
| MMStar | 68.6% | #19 of 173 | Official leaderboard | |
| MTVQA-Test | 30.9% | #2 of 18 | Official leaderboard | |
| Video-MME | 77.8% | #3 of 26 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 69.8% | #27 of 152 | Official leaderboard | |
| MathVista | 70.5% | #8 of 43 | Official leaderboard | |
| VHELM | 71.0 | #8 of 41 | Official leaderboard | |
| CharXiv | 43% | #17 of 77 | Official leaderboard | |
| MEGA-Bench | 47.7% | #5 of 22 | Official leaderboard | |
| MMMU | 64.5% | #47 of 136 | Official leaderboard | |
| MMVU | 53% | #10 of 28 | Official leaderboard | |
| MME-CoT | 56.2% | #4 of 11 | Official leaderboard | |
| MMMU-Pro | 46.2% | #27 of 57 | Official leaderboard | |
| VisualPuzzles | 34.9% | #15 of 29 | Official leaderboard | |
| LVBench | 41.3% | #5 of 9 | Official leaderboard | |
| MATH-Vision (MathVision) | 25.9% | #79 of 105 | Official leaderboard | |
| VISTA | 28.6% | #46 of 54 | Official leaderboard | |
| ZeroBench | 1.00% | #59 of 66 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 74.8 | #13 of 173 | Official leaderboard |