Qwen2-VL-7B-Instruct
Qwen2-VL-7B-Instruct ranks first on VCR (Visual Caption Restoration).
Released 29 Aug 2024
28
Benchmarks with a score
1
Ranked first, out of five or more models
2
In the top three
Unknown lab Aug 2024
Qwen2-VL-7B-Instruct
No Epoch Capabilities Index score yet
- OCRBench v2 42.3% #37 /65
- VisualPuzzles 30.2% #21 /29
- HumanEval-V 1.60% #18 /24
Where it ranks
Its best score on each benchmark, and where that ranks among every model with a score there. We rank each model once, at its best setting.
Knowledge and language
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| MMBench-CN v1.1 | 80.3% | #43 of 178 | Official leaderboard | |
| Vectara Hallucination Leaderboard (HHEM) | 4.20% | #79 of 148 | Official leaderboard | |
| Creation-MMBench | -43.76 | #18 of 19 | Official leaderboard |
Math and science
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| U-MATH | 26.3% | #28 of 30 | Official leaderboard |
Coding
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| HumanEval-V | 1.60% | #18 of 24 | Official leaderboard |
Agents and tools
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| ScreenSpot-Pro | 1.58% | #10 of 11 | Official leaderboard | |
| BALROG | 3.70% | #35 of 35 | Official leaderboard |
Vision, audio and robotics
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| VCR (Visual Caption Restoration) | 75.6% | #1 of 42 | Official leaderboard | |
| A-Bench | 76.4% | #19 of 116 | Official leaderboard | |
| CCBench (MMBench Chinese Culture) | 65.7% | #30 of 152 | Official leaderboard | |
| MMBench | 82.8% | #33 of 155 | Official leaderboard | |
| MMBench-EN v1.1 | 81.8% | #40 of 178 | Official leaderboard | |
| TOMATO | 31.5% | #5 of 22 | Official leaderboard | |
| MTVQA-Test | 25.6% | #5 of 18 | Official leaderboard | |
| OVBench | 49.7% | #2 of 7 | Official leaderboard | |
| MMStar | 60.7% | #57 of 173 | Official leaderboard | |
| MEGA-Bench | 36% | #9 of 22 | Official leaderboard | |
| CharXiv | 34.6% | #33 of 77 | Official leaderboard | |
| VHELM | 53.4 | #19 of 41 | Official leaderboard | |
| Video-MME | 63.9% | #24 of 50 | Epoch AI | |
| OCRBench v2 | 42.3% | #37 of 65 | Official leaderboard | |
| MME-CoT | 42.1% | #7 of 11 | Official leaderboard | |
| MMVU | 42.1% | #18 of 28 | Official leaderboard | |
| ERQA | 34.4% | #12 of 17 | Official leaderboard | |
| VisualPuzzles | 30.2% | #21 of 29 | Official leaderboard | |
| Vision Arena | 1,031 | #123 of 136 | Official leaderboard | |
| MATH-Vision (MathVision) | 16.3% | #95 of 105 | Official leaderboard |
Other
| Benchmark | Score | Rank | Source | |
|---|---|---|---|---|
| OpenVLM Leaderboard | 67.1 | #49 of 173 | Official leaderboard |