Benchmark Almanac

A-Bench

Qwen2.5-VL-72B 81% leads since Jan 2025 Not in model cards

Models answer multiple-choice questions about AI-generated images, covering both meaning and low-level visual quality, to test them as judges of image generators.

The OpenCompass OpenVLM leaderboard lists it for 160 models. A sibling of Q-Bench from the same group.

In the authors’ words

How to accurately and efficiently assess AI-generated images (AIGIs) remains a critical challenge for generative models. Given the high costs and extensive time commitments required for user studies, many researchers have turned towards employing large multi-modal models (LMMs) as AIGI evaluators, the precision and validity of which are still questionable. Furthermore, traditional benchmarks often utilize mostly natural-captured content rather than AIGIs to test the abilities of LMMs, leading to a noticeable gap for AIGIs. Therefore, we introduce A-Bench in this paper, a benchmark designed to diagnose whether LMMs are masters at evaluating AIGIs. Specifically, A-Bench is organized under two key principles: 1) Emphasizing both high-level semantic understanding and low-level visual quality perception to address the intricate demands of AIGIs. 2) Various generative models are utilized for AIGI creation, and various LMMs are employed for evaluation, which ensures a comprehensive validation scope. Ultimately, 2,864 AIGIs from 16 text-to-image models are sampled, each paired with question-answers annotated by human experts, and tested across 18 leading LMMs. We hope that A-Bench will significantly enhance the evaluation process and promote the generation quality for AIGIs. The benchmark is available at https://github.com/Q-Future/A-Bench.

Abstract of A-Bench: Are LMMs Masters at Evaluating AI-generated Images? (arXiv:2406.03070, CC0)
Made by Q-Future (Nanyang Technological University and Shanghai Jiao Tong University) Released Jun 2024 General visual questions All tasks public Last model added Jun 2025 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
Q-Future/A-Bench 117 stars Paper Leaderboard
158
Entries on its own leaderboard, 116 models we track

Qwen2.5-VL-72B leads with 81% among models released by Jun 2025

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in Jun 2025, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

158 entries. Scroll sideways for the rest.

  1. 81.0 Qwen2.5-VL-72B
  2. 79.6 GPT-4.1
  3. 79.4 Qwen2-VL-72B-Instruct
  4. 79.4 RBDash-v1.5
  5. 79.4 InternVL2.5-26B-MPO
  6. 79.4 Ovis2-16B
  7. 79.3 InternVL2.5-78B-MPO
  8. 79.2 GPT-4o
  9. 79.1 GPT-4o
  10. 78.9 Qwen-VL-Max-0809
  11. 78.9 InternVL3-78B
  12. 78.8 Taiyi
  13. 78.7 Ovis2-34B
  14. 78.6 InternVL2.5-38B-MPO
  15. 78.5 MUG-U-7B
  16. 78.4 Step-1o
  17. 77.7 BlueLM-V-3B
  18. 77.5 MiniCPM-o-2_6
  19. 77.5 Ovis2-4B
  20. 77.4 GPT-4.1 mini
  21. 77.1 InternVL3-38B
  22. 77.0 InternVL2.5-8B-MPO
  23. 76.6 Ovis2-2B
  24. 76.5 Ovis1.6-Gemma2-27B
  25. 76.4 Qwen2-VL-7B-Instruct
  26. 76.4 Ovis2-8B
  27. 76.3 Grok 2 Vision (1212)
  28. 76.3 Gemma 3 27B
  29. 76.3 InternVL3-14B
  30. 76.3 InternVL3-8B
  31. 76.1 POINTS-Qwen2.5-7B
  32. 76.1 POINTS1.5-Qwen2.5-7B
  33. 76.1 Kimi-VL-A3B-Instruct
  34. 75.9 InternVL2.5-4B-MPO
  35. 75.8 BailingMM-Pro-0120
  36. 75.7 HunYuan-Standard-Vision
  37. 75.7 Qwen2.5-VL-7B
  38. 75.4 Gemini 1.5 Pro
  39. 75.4 Qwen-VL-Plus-0809
  40. 75.4 Aquila-VL-2B
  41. 75.4 abab7-preview
  42. 75.4 Phi-4-multimodal
  43. 75.2 InternVL2-40B
  44. 75.2 LLaVA-OneVision 7B
  45. 75.2 Ovis1.6-Llama3.2-3B
  46. 75.1 Ovis1.5-Llama3-8B
  47. 75.0 POINTS-Yi-1.5-9B
  48. 74.9 GLM-4v-Plus-20250111
  49. 74.9 Qwen2.5-VL-3B
  50. 74.6 MiniCPM-V-2_6
  51. 74.5 SAIL-VL-2B
  52. 74.4 Ovis1.6-Gemma2-9B
  53. 74.4 VARCO-VISION-14B
  54. 74.3 InternVL2-26B
  55. 74.3 moonshot-v1-8k
  56. 74.2 Ross-Qwen2-7B
  57. 74.2 InternVL3-2B
  58. 74.0 GLM-4V-9B
  59. 73.9 InternVL2-Llama3-76B
  60. 73.8 Molmo-7B-D
  61. 73.8 SmolVLM2-2.2B-Instruct
  62. 73.7 Eagle-X5-34B-Chat
  63. 73.6 Gemini 1.5 Flash
  64. 73.5 Ovis1.5-Gemma2-9B
  65. 73.5 LLaVA-OneVision 7B (SI)
  66. 73.3 MiniCPM-Llama3-V 2.5
  67. 73.1 Pixtral 12B
  68. 72.9 InternVL2-8B
  69. 72.9 Mantis-8B-Idefics2
  70. 72.9 GPT-4.1 nano
  71. 72.7 WeMM
  72. 72.6 Bunny-Llama-3-8B-V
  73. 72.6 XinYuan-VL-2B-Instruct
  74. 72.6 Granite Vision 3.3 2B
  75. 72.4 InternVL2-8B-MPO
  76. 72.4 Granite Vision 3.2 2B
  77. 72.3 CogVLM2-19B-Chat
  78. 72.3 Cambrian-1-34B
  79. 72.1 NVLM-D-72B
  80. 71.7 InternVL-Chat-V1-5
  81. 71.7 Vintern-3B-beta
  82. 71.5 MMAlaya2
  83. 71.4 Step-1.5V-mini
  84. 71.3 Qwen2-VL-2B-Instruct
  85. 71.3 Aria
  86. 71.3 Ovis2-1B
  87. 71.1 Mantis-8B-siglip-llama3
  88. 71.1 InternVL2.5-2B-MPO
  89. 70.8 Eagle-X5-13B
  90. 70.7 Mantis-8B-clip-llama3
  91. 70.7 H2OVL-2B
  92. 70.7 Claude 3.7 Sonnet
  93. 70.6 InternVL2-4B
  94. 70.5 Idefics2-8B
  95. 70.5 Slime-8B
  96. 70.4 GPT-4V
  97. 70.4 TransCore-M
  98. 70.3 360VL-70B
  99. 70.3 BailingMM-Lite-1203
  100. 70.3 VITA-1.5
  101. 70.3 Gemma 3 12B
  102. 70.2 LLaVA-NeXT-Interleave-7B-DPO
  103. 70.0 Yi-VL-34B
  104. 69.8 Yi-VL-6B
  105. 69.7 InternLM-XComposer2
  106. 69.7 llama3-llava-next-8b
  107. 69.6 Mini-InternVL-Chat-4B-V1-5
  108. 69.2 Llama 3.2 11B
  109. 69.1 DeepSeek-VL-7B-Chat
  110. 68.9 LLaVA-NeXT-Interleave-7B
  111. 68.9 InternVL3-1B
  112. 68.6 Cambrian-1-8B
  113. 68.4 Eagle-X5-7B
  114. 68.4 Janus-Pro-7B
  115. 68.3 llava-v1.6-mistral-7b
  116. 68.2 Phi-3-vision-128k-instruct
  117. 68.0 MiniCPM-V 2.0
  118. 67.6 LLaVA-OneVision 0.5B
  119. 67.5 InternVL2.5-1B-MPO
  120. 67.3 Cambrian-1-13B
  121. 67.0 Gemma 3 4B
  122. 66.9 CogVLM-17B-Chat
  123. 66.2 Slime-7B
  124. 66.1 Mini-InternVL-Chat-2B-V1-5
  125. 66.0 Molmo-7B-O
  126. 65.8 llava-v1.6-vicuna-7b
  127. 65.6 llava-v1.5-7b
  128. 65.1 MiniMonkey
  129. 65.0 SmolVLM-Instruct
  130. 64.8 LLaVA-v1.5-13B
  131. 64.7 SmolVLM-Instruct-DPO
  132. 64.3 llava-v1.6-vicuna-13b
  133. 64.2 InternVL2-2B
  134. 64.0 InternVL2-1B
  135. 63.9 InternLM-XComposer2-VL-1.8B
  136. 63.5 Falcon2-11B-VLM
  137. 63.3 MolmoE-1B
  138. 63.2 Monkey-Chat
  139. 63.1 Qwen-VL-Chat
  140. 63.1 H2OVL-800M
  141. 63.0 Slime-13B
  142. 62.8 DeepSeek-VL-1.3B-Chat
  143. 62.4 AKI-4B
  144. 61.9 Monkey
  145. 61.2 Emu3-Chat
  146. 61.2 abab6.5s
  147. 61.2 SmolVLM2-500M-Video-Instruct
  148. 60.3 Mantis-8B-Fuyu
  149. 59.8 PaliGemma-3B-mix-448
  150. 59.6 Moondream1
  151. 59.1 Janus-1.3B
  152. 58.9 SmolVLM-Synthetic
  153. 58.8 LLaVA-OneVision 0.5B (SI)
  154. 54.7 SmolVLM-500M-Instruct
  155. 52.2 Qwen-VL
  156. 45.3 SmolVLM2-256M-Video-Instruct
  157. 36.1 SmolVLM-256M-Instruct
  158. 33.2 Kosmos-2
  1. 1 Qwen2.5-VL-72B 81%
  2. 2 GPT-4.1 79.6%
  3. 3 Qwen2-VL-72B-Instruct 79.4%
  4. 4 RBDash-v1.5 79.4%
  5. 5 InternVL2.5-26B-MPO 79.4%
  6. 6 Ovis2-16B 79.4%
  7. 7 InternVL2.5-78B-MPO 79.3%
  8. 8 GPT-4o May 2024, detail-high 79.2%
  9. 9 GPT-4o Aug 2024, detail-high 79.1%
  10. 10 Qwen-VL-Max-0809 78.9%
  11. 11 InternVL3-78B 78.9%
  12. 12 Taiyi 78.8%
  13. 13 Ovis2-34B 78.7%
  14. 14 InternVL2.5-38B-MPO 78.6%
  15. 15 MUG-U-7B 78.5%
  16. 16 Step-1o 78.4%
  17. 17 BlueLM-V-3B 77.7%
  18. 18 MiniCPM-o-2_6 77.5%
  19. 19 Ovis2-4B 77.5%
  20. 20 GPT-4.1 mini 77.4%
  21. 21 InternVL3-38B 77.1%
  22. 22 InternVL2.5-8B-MPO 77%
  23. 23 Ovis2-2B 76.6%
  24. 24 Ovis1.6-Gemma2-27B 76.5%
  25. 25 Qwen2-VL-7B-Instruct 76.4%
  26. 26 Ovis2-8B 76.4%
  27. 27 Grok 2 Vision (1212) 76.3%
  28. 28 Gemma 3 27B 76.3%
  29. 29 InternVL3-14B 76.3%
  30. 30 InternVL3-8B 76.3%
  31. 31 POINTS-Qwen2.5-7B 76.1%
  32. 32 POINTS1.5-Qwen2.5-7B 76.1%
  33. 33 Kimi-VL-A3B-Instruct 76.1%
  34. 34 InternVL2.5-4B-MPO 75.9%
  35. 35 BailingMM-Pro-0120 75.8%
  36. 36 HunYuan-Standard-Vision 75.7%
  37. 37 Qwen2.5-VL-7B 75.7%
  38. 38 Gemini 1.5 Pro May 2024 75.4%
  39. 39 Qwen-VL-Plus-0809 75.4%
  40. 40 Aquila-VL-2B 75.4%
  41. 41 abab7-preview 75.4%
  42. 42 Phi-4-multimodal 75.4%
  43. 43 InternVL2-40B 75.2%
  44. 44 LLaVA-OneVision 7B 75.2%
  45. 45 Ovis1.6-Llama3.2-3B 75.2%
  46. 46 Ovis1.5-Llama3-8B 75.1%
  47. 47 POINTS-Yi-1.5-9B 75%
  48. 48 GLM-4v-Plus-20250111 74.9%
  49. 49 Qwen2.5-VL-3B 74.9%
  50. 50 MiniCPM-V-2_6 74.6%
  51. 51 SAIL-VL-2B 74.5%
  52. 52 Ovis1.6-Gemma2-9B 74.4%
  53. 53 VARCO-VISION-14B 74.4%
  54. 54 InternVL2-26B 74.3%
  55. 55 moonshot-v1-8k 74.3%
  56. 56 Ross-Qwen2-7B 74.2%
  57. 57 InternVL3-2B 74.2%
  58. 58 GLM-4V-9B 74%
  59. 59 InternVL2-Llama3-76B 73.9%
  60. 60 Molmo-7B-D 73.8%
  61. 61 SmolVLM2-2.2B-Instruct 73.8%
  62. 62 Eagle-X5-34B-Chat 73.7%
  63. 63 Gemini 1.5 Flash Sep 2024 73.6%
  64. 64 Ovis1.5-Gemma2-9B 73.5%
  65. 65 LLaVA-OneVision 7B (SI) 73.5%
  66. 66 MiniCPM-Llama3-V 2.5 73.3%
  67. 67 Pixtral 12B 73.1%
  68. 68 InternVL2-8B 72.9%
  69. 69 Mantis-8B-Idefics2 72.9%
  70. 70 GPT-4.1 nano 72.9%
  71. 71 WeMM 72.7%
  72. 72 Bunny-Llama-3-8B-V 72.6%
  73. 73 XinYuan-VL-2B-Instruct 72.6%
  74. 74 Granite Vision 3.3 2B 72.6%
  75. 75 InternVL2-8B-MPO 72.4%
  76. 76 Granite Vision 3.2 2B 72.4%
  77. 77 CogVLM2-19B-Chat 72.3%
  78. 78 Cambrian-1-34B 72.3%
  79. 79 NVLM-D-72B 72.1%
  80. 80 InternVL-Chat-V1-5 71.7%
  81. 81 Vintern-3B-beta 71.7%
  82. 82 MMAlaya2 71.5%
  83. 83 Step-1.5V-mini 71.4%
  84. 84 Qwen2-VL-2B-Instruct 71.3%
  85. 85 Aria 71.3%
  86. 86 Ovis2-1B 71.3%
  87. 87 Mantis-8B-siglip-llama3 71.1%
  88. 88 InternVL2.5-2B-MPO 71.1%
  89. 89 Eagle-X5-13B 70.8%
  90. 90 Mantis-8B-clip-llama3 70.7%
  91. 91 H2OVL-2B 70.7%
  92. 92 Claude 3.7 Sonnet 70.7%
  93. 93 InternVL2-4B 70.6%
  94. 94 Idefics2-8B 70.5%
  95. 95 Slime-8B 70.5%
  96. 96 GPT-4V detail-high 70.4%
  97. 97 TransCore-M 70.4%
  98. 98 360VL-70B 70.3%
  99. 99 BailingMM-Lite-1203 70.3%
  100. 100 VITA-1.5 70.3%
  101. 101 Gemma 3 12B 70.3%
  102. 102 LLaVA-NeXT-Interleave-7B-DPO 70.2%
  103. 103 Yi-VL-34B 70%
  104. 104 Yi-VL-6B 69.8%
  105. 105 InternLM-XComposer2 69.7%
  106. 106 llama3-llava-next-8b 69.7%
  107. 107 Mini-InternVL-Chat-4B-V1-5 69.6%
  108. 108 Llama 3.2 11B 69.2%
  109. 109 DeepSeek-VL-7B-Chat 69.1%
  110. 110 LLaVA-NeXT-Interleave-7B 68.9%
  111. 111 InternVL3-1B 68.9%
  112. 112 Cambrian-1-8B 68.6%
  113. 113 Eagle-X5-7B 68.4%
  114. 114 Janus-Pro-7B 68.4%
  115. 115 llava-v1.6-mistral-7b 68.3%
  116. 116 Phi-3-vision-128k-instruct 68.2%
  117. 117 MiniCPM-V 2.0 68%
  118. 118 LLaVA-OneVision 0.5B 67.6%
  119. 119 InternVL2.5-1B-MPO 67.5%
  120. 120 Cambrian-1-13B 67.3%
  121. 121 Gemma 3 4B 67%
  122. 122 CogVLM-17B-Chat 66.9%
  123. 123 Slime-7B 66.2%
  124. 124 Mini-InternVL-Chat-2B-V1-5 66.1%
  125. 125 Molmo-7B-O 66%
  126. 126 llava-v1.6-vicuna-7b 65.8%
  127. 127 llava-v1.5-7b 65.6%
  128. 128 MiniMonkey 65.1%
  129. 129 SmolVLM-Instruct 65%
  130. 130 LLaVA-v1.5-13B 64.8%
  131. 131 SmolVLM-Instruct-DPO 64.7%
  132. 132 llava-v1.6-vicuna-13b 64.3%
  133. 133 InternVL2-2B 64.2%
  134. 134 InternVL2-1B 64%
  135. 135 InternLM-XComposer2-VL-1.8B 63.9%
  136. 136 Falcon2-11B-VLM 63.5%
  137. 137 MolmoE-1B 63.3%
  138. 138 Monkey-Chat 63.2%
  139. 139 Qwen-VL-Chat 63.1%
  140. 140 H2OVL-800M 63.1%
  141. 141 Slime-13B 63%
  142. 142 DeepSeek-VL-1.3B-Chat 62.8%
  143. 143 AKI-4B 62.4%
  144. 144 Monkey 61.9%
  145. 145 Emu3-Chat 61.2%
  146. 146 abab6.5s 61.2%
  147. 147 SmolVLM2-500M-Video-Instruct 61.2%
  148. 148 Mantis-8B-Fuyu 60.3%
  149. 149 PaliGemma-3B-mix-448 59.8%
  150. 150 Moondream1 59.6%
  151. 151 Janus-1.3B 59.1%
  152. 152 SmolVLM-Synthetic 58.9%
  153. 153 LLaVA-OneVision 0.5B (SI) 58.8%
  154. 154 SmolVLM-500M-Instruct 54.7%
  155. 155 Qwen-VL 52.2%
  156. 156 SmolVLM2-256M-Video-Instruct 45.3%
  157. 157 SmolVLM-256M-Instruct 36.1%
  158. 158 Kosmos-2 33.2%
#EntryScoreSettingDateSource
1 Qwen2.5-VL-72B 81% Jan 2025 Official leaderboard
2 GPT-4.1 79.6% Apr 2025 Official leaderboard
3 Qwen2-VL-72B-Instruct 79.4% Aug 2024 Official leaderboard
4 RBDash-v1.5 79.4% Nov 2024 Official leaderboard
5 InternVL2.5-26B-MPO 79.4% Dec 2024 Official leaderboard
6 Ovis2-16B 79.4% Feb 2025 Official leaderboard
7 InternVL2.5-78B-MPO 79.3% Dec 2024 Official leaderboard
8 GPT-4o May 2024 79.2% detail-high May 2024 Official leaderboard
9 GPT-4o Aug 2024 79.1% detail-high Aug 2024 Official leaderboard
10 Qwen-VL-Max-0809 78.9% Sep 2024 Official leaderboard
11 InternVL3-78B 78.9% Apr 2025 Official leaderboard
12 Taiyi 78.8% Dec 2024 Official leaderboard
13 Ovis2-34B 78.7% Feb 2025 Official leaderboard
14 InternVL2.5-38B-MPO 78.6% Dec 2024 Official leaderboard
15 MUG-U-7B 78.5% Mar 2025 Official leaderboard
16 Step-1o 78.4% Jan 2025 Official leaderboard
17 BlueLM-V-3B 77.7% Oct 2024 Official leaderboard
18 MiniCPM-o-2_6 77.5% Jan 2025 Official leaderboard
19 Ovis2-4B 77.5% Feb 2025 Official leaderboard
20 GPT-4.1 mini 77.4% Apr 2025 Official leaderboard
21 InternVL3-38B 77.1% Apr 2025 Official leaderboard
22 InternVL2.5-8B-MPO 77% Dec 2024 Official leaderboard
23 Ovis2-2B 76.6% Feb 2025 Official leaderboard
24 Ovis1.6-Gemma2-27B 76.5% Nov 2024 Official leaderboard
25 Qwen2-VL-7B-Instruct 76.4% Aug 2024 Official leaderboard
26 Ovis2-8B 76.4% Feb 2025 Official leaderboard
27 Grok 2 Vision (1212) 76.3% Dec 2024 Official leaderboard
28 Gemma 3 27B 76.3% Mar 2025 Official leaderboard
29 InternVL3-14B 76.3% Apr 2025 Official leaderboard
30 InternVL3-8B 76.3% Apr 2025 Official leaderboard
31 POINTS-Qwen2.5-7B 76.1% Oct 2024 Official leaderboard
32 POINTS1.5-Qwen2.5-7B 76.1% Dec 2024 Official leaderboard
33 Kimi-VL-A3B-Instruct 76.1% Apr 2025 Official leaderboard
34 InternVL2.5-4B-MPO 75.9% Dec 2024 Official leaderboard
35 BailingMM-Pro-0120 75.8% Jan 2025 Official leaderboard
36 HunYuan-Standard-Vision 75.7% Jan 2025 Official leaderboard
37 Qwen2.5-VL-7B 75.7% Jan 2025 Official leaderboard
38 Gemini 1.5 Pro May 2024 75.4% May 2024 Official leaderboard
39 Qwen-VL-Plus-0809 75.4% Sep 2024 Official leaderboard
40 Aquila-VL-2B 75.4% Oct 2024 Official leaderboard
41 abab7-preview 75.4% Dec 2024 Official leaderboard
42 Phi-4-multimodal 75.4% Feb 2025 Official leaderboard
43 InternVL2-40B 75.2% Jul 2024 Official leaderboard
44 LLaVA-OneVision 7B 75.2% Aug 2024 Official leaderboard
45 Ovis1.6-Llama3.2-3B 75.2% Oct 2024 Official leaderboard
46 Ovis1.5-Llama3-8B 75.1% Jul 2024 Official leaderboard
47 POINTS-Yi-1.5-9B 75% Oct 2024 Official leaderboard
48 GLM-4v-Plus-20250111 74.9% Jan 2025 Official leaderboard
49 Qwen2.5-VL-3B 74.9% Jan 2025 Official leaderboard
50 MiniCPM-V-2_6 74.6% Aug 2024 Official leaderboard
51 SAIL-VL-2B 74.5% Dec 2024 Official leaderboard
52 Ovis1.6-Gemma2-9B 74.4% Sep 2024 Official leaderboard
53 VARCO-VISION-14B 74.4% Jan 2025 Official leaderboard
54 InternVL2-26B 74.3% Jul 2024 Official leaderboard
55 moonshot-v1-8k 74.3% Mar 2025 Official leaderboard
56 Ross-Qwen2-7B 74.2% Jan 2025 Official leaderboard
57 InternVL3-2B 74.2% Apr 2025 Official leaderboard
58 GLM-4V-9B 74% Jun 2024 Official leaderboard
59 InternVL2-Llama3-76B 73.9% Jul 2024 Official leaderboard
60 Molmo-7B-D 73.8% Sep 2024 Official leaderboard
61 SmolVLM2-2.2B-Instruct 73.8% Feb 2025 Official leaderboard
62 Eagle-X5-34B-Chat 73.7% Sep 2024 Official leaderboard
63 Gemini 1.5 Flash Sep 2024 73.6% Aug 2024 Official leaderboard
64 Ovis1.5-Gemma2-9B 73.5% Jul 2024 Official leaderboard
65 LLaVA-OneVision 7B (SI) 73.5% Aug 2024 Official leaderboard
66 MiniCPM-Llama3-V 2.5 73.3% May 2024 Official leaderboard
67 Pixtral 12B 73.1% Sep 2024 Official leaderboard
68 InternVL2-8B 72.9% Jul 2024 Official leaderboard
69 Mantis-8B-Idefics2 72.9% Aug 2024 Official leaderboard
70 GPT-4.1 nano 72.9% Apr 2025 Official leaderboard
71 WeMM 72.7% Jun 2024 Official leaderboard
72 Bunny-Llama-3-8B-V 72.6% Apr 2024 Official leaderboard
73 XinYuan-VL-2B-Instruct 72.6% Sep 2024 Official leaderboard
74 Granite Vision 3.3 2B 72.6% Jun 2025 Official leaderboard
75 InternVL2-8B-MPO 72.4% Nov 2024 Official leaderboard
76 Granite Vision 3.2 2B 72.4% Feb 2025 Official leaderboard
77 CogVLM2-19B-Chat 72.3% May 2024 Official leaderboard
78 Cambrian-1-34B 72.3% Jun 2024 Official leaderboard
79 NVLM-D-72B 72.1% Sep 2024 Official leaderboard
80 InternVL-Chat-V1-5 71.7% Apr 2024 Official leaderboard
81 Vintern-3B-beta 71.7% Oct 2024 Official leaderboard
82 MMAlaya2 71.5% Aug 2024 Official leaderboard
83 Step-1.5V-mini 71.4% Dec 2024 Official leaderboard
84 Qwen2-VL-2B-Instruct 71.3% Aug 2024 Official leaderboard
85 Aria 71.3% Sep 2024 Official leaderboard
86 Ovis2-1B 71.3% Feb 2025 Official leaderboard
87 Mantis-8B-siglip-llama3 71.1% Aug 2024 Official leaderboard
88 InternVL2.5-2B-MPO 71.1% Dec 2024 Official leaderboard
89 Eagle-X5-13B 70.8% Aug 2024 Official leaderboard
90 Mantis-8B-clip-llama3 70.7% Aug 2024 Official leaderboard
91 H2OVL-2B 70.7% Oct 2024 Official leaderboard
92 Claude 3.7 Sonnet 70.7% Feb 2025 Official leaderboard
93 InternVL2-4B 70.6% Jul 2024 Official leaderboard
94 Idefics2-8B 70.5% Apr 2024 Official leaderboard
95 Slime-8B 70.5% Sep 2024 Official leaderboard
96 GPT-4V 70.4% detail-high Nov 2023 Official leaderboard
97 TransCore-M 70.4% Apr 2024 Official leaderboard
98 360VL-70B 70.3% May 2024 Official leaderboard
99 BailingMM-Lite-1203 70.3% Dec 2024 Official leaderboard
100 VITA-1.5 70.3% Dec 2024 Official leaderboard
101 Gemma 3 12B 70.3% Mar 2025 Official leaderboard
102 LLaVA-NeXT-Interleave-7B-DPO 70.2% Jun 2024 Official leaderboard
103 Yi-VL-34B 70% Dec 2023 Official leaderboard
104 Yi-VL-6B 69.8% Dec 2023 Official leaderboard
105 InternLM-XComposer2 69.7% Jan 2024 Official leaderboard
106 llama3-llava-next-8b 69.7% Apr 2024 Official leaderboard
107 Mini-InternVL-Chat-4B-V1-5 69.6% May 2024 Official leaderboard
108 Llama 3.2 11B 69.2% Sep 2024 Official leaderboard
109 DeepSeek-VL-7B-Chat 69.1% Mar 2024 Official leaderboard
110 LLaVA-NeXT-Interleave-7B 68.9% Jun 2024 Official leaderboard
111 InternVL3-1B 68.9% Apr 2025 Official leaderboard
112 Cambrian-1-8B 68.6% Jun 2024 Official leaderboard
113 Eagle-X5-7B 68.4% Aug 2024 Official leaderboard
114 Janus-Pro-7B 68.4% Jan 2025 Official leaderboard
115 llava-v1.6-mistral-7b 68.3% Jan 2024 Official leaderboard
116 Phi-3-vision-128k-instruct 68.2% May 2024 Official leaderboard
117 MiniCPM-V 2.0 68% Apr 2024 Official leaderboard
118 LLaVA-OneVision 0.5B 67.6% Aug 2024 Official leaderboard
119 InternVL2.5-1B-MPO 67.5% Dec 2024 Official leaderboard
120 Cambrian-1-13B 67.3% Jun 2024 Official leaderboard
121 Gemma 3 4B 67% Mar 2025 Official leaderboard
122 CogVLM-17B-Chat 66.9% Nov 2023 Official leaderboard
123 Slime-7B 66.2% Sep 2024 Official leaderboard
124 Mini-InternVL-Chat-2B-V1-5 66.1% May 2024 Official leaderboard
125 Molmo-7B-O 66% Sep 2024 Official leaderboard
126 llava-v1.6-vicuna-7b 65.8% Jan 2024 Official leaderboard
127 llava-v1.5-7b 65.6% Oct 2023 Official leaderboard
128 MiniMonkey 65.1% Sep 2024 Official leaderboard
129 SmolVLM-Instruct 65% Nov 2024 Official leaderboard
130 LLaVA-v1.5-13B 64.8% Oct 2023 Official leaderboard
131 SmolVLM-Instruct-DPO 64.7% Jan 2025 Official leaderboard
132 llava-v1.6-vicuna-13b 64.3% Jan 2024 Official leaderboard
133 InternVL2-2B 64.2% Jul 2024 Official leaderboard
134 InternVL2-1B 64% Jul 2024 Official leaderboard
135 InternLM-XComposer2-VL-1.8B 63.9% Apr 2024 Official leaderboard
136 Falcon2-11B-VLM 63.5% May 2024 Official leaderboard
137 MolmoE-1B 63.3% Sep 2024 Official leaderboard
138 Monkey-Chat 63.2% Jan 2024 Official leaderboard
139 Qwen-VL-Chat 63.1% Aug 2023 Official leaderboard
140 H2OVL-800M 63.1% Oct 2024 Official leaderboard
141 Slime-13B 63% Sep 2024 Official leaderboard
142 DeepSeek-VL-1.3B-Chat 62.8% Mar 2024 Official leaderboard
143 AKI-4B 62.4% Mar 2025 Official leaderboard
144 Monkey 61.9% Nov 2023 Official leaderboard
145 Emu3-Chat 61.2% Sep 2024 Official leaderboard
146 abab6.5s 61.2% Dec 2024 Official leaderboard
147 SmolVLM2-500M-Video-Instruct 61.2% Feb 2025 Official leaderboard
148 Mantis-8B-Fuyu 60.3% Sep 2024 Official leaderboard
149 PaliGemma-3B-mix-448 59.8% May 2024 Official leaderboard
150 Moondream1 59.6% Sep 2024 Official leaderboard
151 Janus-1.3B 59.1% Oct 2024 Official leaderboard
152 SmolVLM-Synthetic 58.9% Jan 2025 Official leaderboard
153 LLaVA-OneVision 0.5B (SI) 58.8% Aug 2024 Official leaderboard
154 SmolVLM-500M-Instruct 54.7% Jan 2025 Official leaderboard
155 Qwen-VL 52.2% Aug 2023 Official leaderboard
156 SmolVLM2-256M-Video-Instruct 45.3% Feb 2025 Official leaderboard
157 SmolVLM-256M-Instruct 36.1% Jan 2025 Official leaderboard
158 Kosmos-2 33.2% Jun 2023 Official leaderboard

3 models have held number one since its release in Jun 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Other labs Qwen
Own leaderboard Not tested since Jun 2025
Qwen2.5-VL-72B 81%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2023 2024 2025 Near the ceiling (85%) Qwen2.5-VL-72B Qwen2-VL-72B-Instruct Bunny-Llama-3-8B-V GPT-4V llava-v1.5-7b Kosmos-2 0% 20% 40% 60% 80% 100% 2023 2024 2025 Near the ceiling (85%) Qwen2.5-VL-72B Bunny-Llama-3-8B-V

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%