Benchmark Almanac

MMMU

GPT-5.1 85.4% leads since Nov 2025 In use

11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy.

Near-saturated. The test set answers became public on 2026-02-12, so test scores can now be computed locally.

In the authors’ words

We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering. These questions span 30 subjects and 183 subfields, comprising 30 highly heterogeneous image types, such as charts, diagrams, maps, tables, music sheets, and chemical structures. Unlike existing benchmarks, MMMU focuses on advanced perception and reasoning with domain-specific knowledge, challenging models to perform tasks akin to those faced by experts. The evaluation of 14 open-source LMMs as well as the proprietary GPT-4V(ision) and Gemini highlights the substantial challenges posed by MMMU. Even the advanced GPT-4V and Gemini Ultra only achieve accuracies of 56% and 59% respectively, indicating significant room for improvement. We believe MMMU will stimulate the community to build next-generation multimodal foundation models towards expert artificial general intelligence.

Abstract of MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI (arXiv:2311.16502, CC0)
Made by MMMU team (OSU, Waterloo and others) Released Nov 2023 General visual questions All tasks public Last in a launch post Aug 2026 Last model added Aug 2026 Scores refreshed 2 Nov 2026
Level
University Undergraduates and entry-level professionals
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
MMMU-Benchmark/MMMU 600 stars Paper Dataset Leaderboard
184
Entries on its own leaderboard, 136 models we track
89
Launch posts list it, from 22 labs
51
System cards report it, from 18 labs

GPT-5.1 leads with 85.4% among models released by Mar 2026

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in Mar 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

184 entries. Scroll sideways for the rest.

  1. 85.4 GPT-5.1
  2. 84.6 DreamPRM-1.5 (GPT-5-mini w/ thinking)
  3. 84.2 GPT-5
  4. 84.0 Gemini 2.5 Deep Think
  5. 82.9 o3
  6. 81.6 o4-mini
  7. 80.7 Claude Opus 4.5
  8. 80.1 dots.vlm1
  9. 79.7 Gemini 2.5 Flash
  10. 79.6 Gemini 2.5 Pro
  11. 78.7 Qwen3-VL-235B-A22B
  12. 78.2 o1
  13. 78.0 Grok 3
  14. 77.9 Seed1.5-VL
  15. 77.8 Claude Sonnet 4.5
  16. 77.7 Intern-S1
  17. 77.1 Claude Opus 4.1
  18. 76.5 Claude Opus 4
  19. 76.1 Llama 4 Behemoth
  20. 76.0 Skywork-R1V3-38B
  21. 75.4 GLM-4.5V
  22. 75.0 Claude 3.7 Sonnet
  23. 74.8 Seed1.6-Thinking
  24. 74.4 GPT-4.5
  25. 74.4 Claude Sonnet 4
  26. 73.6 Skywork-R1V2-38B
  27. 73.4 Llama 4 Maverick
  28. 72.7 Gemini 2.0 Pro
  29. 72.6 KiQ-v0
  30. 72.2 InternVL3-78B
  31. 71.9 LLaVA-Critic-R1+
  32. 71.7 Gemini 2.0 Flash
  33. 70.7 GPT-4o
  34. 70.6 MiMo-VL-7B-RL-2508
  35. 70.4 Spark4.0-Turbo
  36. 70.3 QVQ-72B-Preview
  37. 70.1 InternVL2_5-78B
  38. 70.1 InternVL3-38B
  39. 69.7 LLaVA-Critic-R1
  40. 69.4 Llama 4 Scout
  41. 69.1 GPT-4o
  42. 69.0 Skywork-R1V-38B
  43. 68.3 Claude 3.5 Sonnet
  44. 68.0 Gemini 2.0 Flash-Lite
  45. 68.0 Amazon Nova Premier
  46. 68.0 GLM-4.1V-9B-Thinking
  47. 67.1 InternVL3-14B
  48. 66.7 MiMo-VL-7B-RL
  49. 65.8 Gemini 1.5 Pro (0801)
  50. 65.6 Ovis2-34B
  51. 64.9 Gemma 3 27B
  52. 64.6 MiMo-VL-7B-SFT
  53. 64.5 Qwen2-VL-72B-Instruct
  54. 64.5 Qwen2.5-VL-72B
  55. 63.9 InternVL2.5-38B
  56. 62.7 InternVL3-8B
  57. 62.2 Gemini 1.5 Pro
  58. 62.0 InternVL2-Pro
  59. 62.0 Amazon Nova Pro
  60. 61.4 TeleMM
  61. 60.3 Llama 3.2 90B
  62. 60.2 NVLM-H 1.0 78B
  63. 60.0 InternVL2.5-26B
  64. 59.7 NVLM-D-72B
  65. 59.6 Ovis2-16B
  66. 59.6 Gemma 3 12B
  67. 59.4 Gemini 1.0 Ultra
  68. 59.4 Claude 3 Opus
  69. 59.4 GPT-4o mini
  70. 59.0 Ovis2-8B
  71. 58.7 NVILA
  72. 58.6 Qwen2.5-VL-7B
  73. 58.3 InternVL2-Llama3-76B
  74. 58.3 Qianfan-VL-70B
  75. 57.7 RBDash V1.2
  76. 57.7 InternVL3-9B
  77. 57.0 Kimi-VL-A3B
  78. 56.8 GPT-4V
  79. 56.8 LLaVA-OneVision 72B
  80. 56.3 Reka Core
  81. 56.1 Gemini 1.5 Flash
  82. 56.0 InternVL2.5-8B
  83. 55.8 Eagle2.5-8B
  84. 55.2 InternVL2-40B
  85. 54.9 Aria
  86. 54.6 SenseChat-Vision-0423-Preview
  87. 53.8 POINTS1.5
  88. 53.3 Reka Flash
  89. 53.1 Claude 3 Sonnet
  90. 53.1 Qwen2.5-VL-3B
  91. 53.0 EVLM-KTO
  92. 52.0 HPT Pro
  93. 51.6 InternVL-Chat-V1-2
  94. 51.4 Qwen-VL-Max
  95. 51.4 Skywork-VL
  96. 51.2 InternVL2-8B
  97. 51.1 DeepSeek-VL2
  98. 51.0 SpaRE-7B
  99. 50.8 360VL
  100. 50.8 MAmmoTH-VL-8B
  101. 50.7 Llama 3.2 11B
  102. 50.6 BLIP3-o8B
  103. 50.2 Claude 3 Haiku
  104. 49.9 LLaVA-NeXT-72B
  105. 49.8 Ovis1.5-Gemma2-9B
  106. 49.8 MiniCPM-V-2_6
  107. 49.7 Cambrian-1-34B
  108. 48.8 LLaVA-OneVision 7B
  109. 48.8 Gemma 3 4B
  110. 48.6 Ovis-Clip-Llama3-8B
  111. 48.6 InternVL3-2B
  112. 48.3 Adept Fuyu-Heavy
  113. 48.1 LLaVA-v1.6-34B
  114. 48.0 DeepSeek-VL2-Small
  115. 48.0 Ovis2-4B
  116. 47.9 Gemini 1.0 Pro
  117. 47.4 Aquila-VL-2B
  118. 47.1 Qianfan-VL-8B
  119. 46.6 Idefics3-8B-Llama3
  120. 46.6 BLIP3-o4B
  121. 46.4 Qianfan-VL-3B
  122. 46.2 Marco-VL-Plus
  123. 45.9 Yi-VL-34B
  124. 45.8 MiniCPM-Llama3-V 2.5
  125. 45.3 Ovis2-2B
  126. 45.2 Qwen-VL-Plus
  127. 45.1 Aya Vision 32B
  128. 44.2 Mogao-7B
  129. 44.0 HPT Air
  130. 43.6 InternVL2.5-2B
  131. 43.4 InternVL3-1B
  132. 43.3 Bunny-8B
  133. 43.0 InternLM-XComposer2-VL-7B
  134. 43.0 Phi-3.5-vision-instruct
  135. 42.9 Flash-VL-2B-Dynamic
  136. 42.8 Reka Edge
  137. 42.7 Cambrian-1-8B
  138. 41.8 HyViLM
  139. 41.8 MetaMorph
  140. 41.4 Bunny-v1.0-4B
  141. 41.2 Marco-VL
  142. 41.1 OmniLMM-12B
  143. 40.9 InternVL2.5-1B
  144. 40.7 DeepSeek-VL2-Tiny
  145. 40.0 Cambrian-1-13B
  146. 40.0 SpaRE-2B
  147. 39.9 Aya Vision 8B
  148. 39.4 InfiMM-Zephyr-7B
  149. 39.1 Yi-VL-6B
  150. 39.1 InternVL-Chat-V1-1
  151. 38.3 Math-LLaVA-13B
  152. 38.2 Bunny-v1.0-3B
  153. 38.2 ILLUME
  154. 38.2 LLaVA-UHD v2
  155. 38.0 SVIT
  156. 37.2 MiniCPM-V
  157. 37.1 MiniCPM-V 2.0
  158. 36.4 LLaVA-v1.5-13B
  159. 36.3 Emu2-Chat
  160. 36.2 llava-v1.6-vicuna-13b
  161. 36.0 Ovis2-1B
  162. 35.9 Qwen-VL-Chat
  163. 35.8 llava-v1.6-vicuna-7b
  164. 35.7 InstructBLIP FLAN-T5-XXL
  165. 35.4 BLIP-2 FLAN-T5-XXL
  166. 35.3 llava-v1.6-mistral-7b
  167. 34.4 BLIP-2 FLAN-T5-XL
  168. 34.2 SynerGen-VL
  169. 33.7 HoVLE
  170. 33.2 HoVLE (HD)
  171. 32.9 InstructBLIP FLAN-T5-XL
  172. 32.9 SPHINX
  173. 32.8 Olympus
  174. 32.7 mPLUG-Owl2
  175. 32.6 Gemini Nano2
  176. 32.2 Otter
  177. 32.1 CogVLM-17B-Chat
  178. 31.6 Emu3
  179. 29.8 LLaMA-Adapter2-7B
  180. 28.7 OpenFlamingo2-9B
  181. 27.9 Fuyu-8B
  182. 27.4 Show-o
  183. 26.8 MiniGPT4-Vicuna-13B
  184. 24.4 Kosmos-2
  1. 1 GPT-5.1 85.4%
  2. 2 DreamPRM-1.5 (GPT-5-mini w/ thinking) 84.6%
  3. 3 GPT-5 thinking 84.2%
  4. 4 Gemini 2.5 Deep Think 84%
  5. 5 o3 82.9%
  6. 6 o4-mini 81.6%
  7. 7 Claude Opus 4.5 80.7%
  8. 8 dots.vlm1 80.1%
  9. 9 Gemini 2.5 Flash May 2025 79.7%
  10. 10 Gemini 2.5 Pro May 2025 79.6%
  11. 11 Qwen3-VL-235B-A22B 78.7%
  12. 12 o1 78.2%
  13. 13 Grok 3 78%
  14. 14 Seed1.5-VL thinking 77.9%
  15. 15 Claude Sonnet 4.5 77.8%
  16. 16 Intern-S1 77.7%
  17. 17 Claude Opus 4.1 77.1%
  18. 18 Claude Opus 4 76.5%
  19. 19 Llama 4 Behemoth 76.1%
  20. 20 Skywork-R1V3-38B 76%
  21. 21 GLM-4.5V thinking 75.4%
  22. 22 Claude 3.7 Sonnet 75%
  23. 23 Seed1.6-Thinking 74.8%
  24. 24 GPT-4.5 74.4%
  25. 25 Claude Sonnet 4 74.4%
  26. 26 Skywork-R1V2-38B 73.6%
  27. 27 Llama 4 Maverick 73.4%
  28. 28 Gemini 2.0 Pro 72.7%
  29. 29 KiQ-v0 72.6%
  30. 30 InternVL3-78B 72.2%
  31. 31 LLaVA-Critic-R1+ 71.9%
  32. 32 Gemini 2.0 Flash Feb 2025 71.7%
  33. 33 GPT-4o Nov 2024 70.7%
  34. 34 MiMo-VL-7B-RL-2508 thinking 70.6%
  35. 35 Spark4.0-Turbo 70.4%
  36. 36 QVQ-72B-Preview 70.3%
  37. 37 InternVL2_5-78B 70.1%
  38. 38 InternVL3-38B 70.1%
  39. 39 LLaVA-Critic-R1 69.7%
  40. 40 Llama 4 Scout 69.4%
  41. 41 GPT-4o May 2024 69.1%
  42. 42 Skywork-R1V-38B 69%
  43. 43 Claude 3.5 Sonnet 68.3%
  44. 44 Gemini 2.0 Flash-Lite 68%
  45. 45 Amazon Nova Premier 68%
  46. 46 GLM-4.1V-9B-Thinking thinking 68%
  47. 47 InternVL3-14B 67.1%
  48. 48 MiMo-VL-7B-RL 66.7%
  49. 49 Gemini 1.5 Pro (0801) 65.8%
  50. 50 Ovis2-34B 65.6%
  51. 51 Gemma 3 27B 64.9%
  52. 52 MiMo-VL-7B-SFT 64.6%
  53. 53 Qwen2-VL-72B-Instruct 64.5%
  54. 54 Qwen2.5-VL-72B 64.5%
  55. 55 InternVL2.5-38B 63.9%
  56. 56 InternVL3-8B 62.7%
  57. 57 Gemini 1.5 Pro May 2024 62.2%
  58. 58 InternVL2-Pro 62%
  59. 59 Amazon Nova Pro 62%
  60. 60 TeleMM 61.4%
  61. 61 Llama 3.2 90B 60.3%
  62. 62 NVLM-H 1.0 78B 60.2%
  63. 63 InternVL2.5-26B 60%
  64. 64 NVLM-D-72B 59.7%
  65. 65 Ovis2-16B 59.6%
  66. 66 Gemma 3 12B 59.6%
  67. 67 Gemini 1.0 Ultra 59.4%
  68. 68 Claude 3 Opus 59.4%
  69. 69 GPT-4o mini 59.4%
  70. 70 Ovis2-8B 59%
  71. 71 NVILA 58.7%
  72. 72 Qwen2.5-VL-7B 58.6%
  73. 73 InternVL2-Llama3-76B 58.3%
  74. 74 Qianfan-VL-70B 58.3%
  75. 75 RBDash V1.2 57.7%
  76. 76 InternVL3-9B 57.7%
  77. 77 Kimi-VL-A3B 57%
  78. 78 GPT-4V 56.8%
  79. 79 LLaVA-OneVision 72B 56.8%
  80. 80 Reka Core 56.3%
  81. 81 Gemini 1.5 Flash May 2024 56.1%
  82. 82 InternVL2.5-8B 56%
  83. 83 Eagle2.5-8B 55.8%
  84. 84 InternVL2-40B 55.2%
  85. 85 Aria 54.9%
  86. 86 SenseChat-Vision-0423-Preview 54.6%
  87. 87 POINTS1.5 53.8%
  88. 88 Reka Flash 53.3%
  89. 89 Claude 3 Sonnet 53.1%
  90. 90 Qwen2.5-VL-3B 53.1%
  91. 91 EVLM-KTO 53%
  92. 92 HPT Pro 52%
  93. 93 InternVL-Chat-V1-2 51.6%
  94. 94 Qwen-VL-Max 51.4%
  95. 95 Skywork-VL 51.4%
  96. 96 InternVL2-8B 51.2%
  97. 97 DeepSeek-VL2 51.1%
  98. 98 SpaRE-7B 51%
  99. 99 360VL 50.8%
  100. 100 MAmmoTH-VL-8B 50.8%
  101. 101 Llama 3.2 11B 50.7%
  102. 102 BLIP3-o8B 50.6%
  103. 103 Claude 3 Haiku 50.2%
  104. 104 LLaVA-NeXT-72B 49.9%
  105. 105 Ovis1.5-Gemma2-9B 49.8%
  106. 106 MiniCPM-V-2_6 49.8%
  107. 107 Cambrian-1-34B 49.7%
  108. 108 LLaVA-OneVision 7B 48.8%
  109. 109 Gemma 3 4B 48.8%
  110. 110 Ovis-Clip-Llama3-8B 48.6%
  111. 111 InternVL3-2B 48.6%
  112. 112 Adept Fuyu-Heavy 48.3%
  113. 113 LLaVA-v1.6-34B 48.1%
  114. 114 DeepSeek-VL2-Small 48%
  115. 115 Ovis2-4B 48%
  116. 116 Gemini 1.0 Pro 47.9%
  117. 117 Aquila-VL-2B 47.4%
  118. 118 Qianfan-VL-8B 47.1%
  119. 119 Idefics3-8B-Llama3 46.6%
  120. 120 BLIP3-o4B 46.6%
  121. 121 Qianfan-VL-3B 46.4%
  122. 122 Marco-VL-Plus 46.2%
  123. 123 Yi-VL-34B 45.9%
  124. 124 MiniCPM-Llama3-V 2.5 45.8%
  125. 125 Ovis2-2B 45.3%
  126. 126 Qwen-VL-Plus 45.2%
  127. 127 Aya Vision 32B 45.1%
  128. 128 Mogao-7B 44.2%
  129. 129 HPT Air 44%
  130. 130 InternVL2.5-2B 43.6%
  131. 131 InternVL3-1B 43.4%
  132. 132 Bunny-8B 43.3%
  133. 133 InternLM-XComposer2-VL-7B 43%
  134. 134 Phi-3.5-vision-instruct 43%
  135. 135 Flash-VL-2B-Dynamic 42.9%
  136. 136 Reka Edge 42.8%
  137. 137 Cambrian-1-8B 42.7%
  138. 138 HyViLM 41.8%
  139. 139 MetaMorph 41.8%
  140. 140 Bunny-v1.0-4B 41.4%
  141. 141 Marco-VL 41.2%
  142. 142 OmniLMM-12B 41.1%
  143. 143 InternVL2.5-1B 40.9%
  144. 144 DeepSeek-VL2-Tiny 40.7%
  145. 145 Cambrian-1-13B 40%
  146. 146 SpaRE-2B 40%
  147. 147 Aya Vision 8B 39.9%
  148. 148 InfiMM-Zephyr-7B 39.4%
  149. 149 Yi-VL-6B 39.1%
  150. 150 InternVL-Chat-V1-1 39.1%
  151. 151 Math-LLaVA-13B 38.3%
  152. 152 Bunny-v1.0-3B 38.2%
  153. 153 ILLUME 38.2%
  154. 154 LLaVA-UHD v2 38.2%
  155. 155 SVIT 38%
  156. 156 MiniCPM-V 37.2%
  157. 157 MiniCPM-V 2.0 37.1%
  158. 158 LLaVA-v1.5-13B 36.4%
  159. 159 Emu2-Chat 36.3%
  160. 160 llava-v1.6-vicuna-13b 36.2%
  161. 161 Ovis2-1B 36%
  162. 162 Qwen-VL-Chat 35.9%
  163. 163 llava-v1.6-vicuna-7b 35.8%
  164. 164 InstructBLIP FLAN-T5-XXL 35.7%
  165. 165 BLIP-2 FLAN-T5-XXL 35.4%
  166. 166 llava-v1.6-mistral-7b 35.3%
  167. 167 BLIP-2 FLAN-T5-XL 34.4%
  168. 168 SynerGen-VL 34.2%
  169. 169 HoVLE 33.7%
  170. 170 HoVLE (HD) 33.2%
  171. 171 InstructBLIP FLAN-T5-XL 32.9%
  172. 172 SPHINX 32.9%
  173. 173 Olympus 32.8%
  174. 174 mPLUG-Owl2 32.7%
  175. 175 Gemini Nano2 32.6%
  176. 176 Otter 32.2%
  177. 177 CogVLM-17B-Chat 32.1%
  178. 178 Emu3 31.6%
  179. 179 LLaMA-Adapter2-7B 29.8%
  180. 180 OpenFlamingo2-9B 28.7%
  181. 181 Fuyu-8B 27.9%
  182. 182 Show-o 27.4%
  183. 183 MiniGPT4-Vicuna-13B 26.8%
  184. 184 Kosmos-2 24.4%
#EntryScoreSettingDateSource
1 GPT-5.1 85.4% Nov 2025 Official leaderboard
2 DreamPRM-1.5 (GPT-5-mini w/ thinking) 84.6% Sep 2025 Official leaderboard
3 GPT-5 84.2% thinking Aug 2025 Official leaderboard
4 Gemini 2.5 Deep Think 84% Aug 2025 Official leaderboard
5 o3 82.9% Apr 2025 Official leaderboard
6 o4-mini 81.6% Apr 2025 Official leaderboard
7 Claude Opus 4.5 80.7% Nov 2025 Official leaderboard
8 dots.vlm1 80.1% Aug 2025 Official leaderboard
9 Gemini 2.5 Flash May 2025 79.7% May 2025 Official leaderboard
10 Gemini 2.5 Pro May 2025 79.6% May 2025 Official leaderboard
11 Qwen3-VL-235B-A22B 78.7% Sep 2025 Official leaderboard
12 o1 78.2% Dec 2024 Official leaderboard
13 Grok 3 78% Apr 2025 Official leaderboard
14 Seed1.5-VL 77.9% thinking May 2025 Official leaderboard
15 Claude Sonnet 4.5 77.8% Sep 2025 Official leaderboard
16 Intern-S1 77.7% Jul 2025 Official leaderboard
17 Claude Opus 4.1 77.1% Aug 2025 Official leaderboard
18 Claude Opus 4 76.5% May 2025 Official leaderboard
19 Llama 4 Behemoth 76.1% Apr 2025 Official leaderboard
20 Skywork-R1V3-38B 76% Jul 2025 Official leaderboard
21 GLM-4.5V 75.4% thinking Aug 2025 Official leaderboard
22 Claude 3.7 Sonnet 75% Feb 2025 Official leaderboard
23 Seed1.6-Thinking 74.8% Jun 2025 Official leaderboard
24 GPT-4.5 74.4% Feb 2025 Official leaderboard
25 Claude Sonnet 4 74.4% May 2025 Official leaderboard
26 Skywork-R1V2-38B 73.6% Apr 2025 Official leaderboard
27 Llama 4 Maverick 73.4% Apr 2025 Official leaderboard
28 Gemini 2.0 Pro 72.7% Feb 2025 Official leaderboard
29 KiQ-v0 72.6% Aug 2025 Official leaderboard
30 InternVL3-78B 72.2% Apr 2025 Official leaderboard
31 LLaVA-Critic-R1+ 71.9% Aug 2025 Official leaderboard
32 Gemini 2.0 Flash Feb 2025 71.7% Feb 2025 Official leaderboard
33 GPT-4o Nov 2024 70.7% Nov 2024 Official leaderboard
34 MiMo-VL-7B-RL-2508 70.6% thinking Aug 2025 Official leaderboard
35 Spark4.0-Turbo 70.4% Jan 2025 Official leaderboard
36 QVQ-72B-Preview 70.3% Dec 2024 Official leaderboard
37 InternVL2_5-78B 70.1% Dec 2024 Official leaderboard
38 InternVL3-38B 70.1% Apr 2025 Official leaderboard
39 LLaVA-Critic-R1 69.7% Aug 2025 Official leaderboard
40 Llama 4 Scout 69.4% Apr 2025 Official leaderboard
41 GPT-4o May 2024 69.1% May 2024 Official leaderboard
42 Skywork-R1V-38B 69% Mar 2025 Official leaderboard
43 Claude 3.5 Sonnet 68.3% Jun 2024 Official leaderboard
44 Gemini 2.0 Flash-Lite 68% Feb 2025 Official leaderboard
45 Amazon Nova Premier 68% Apr 2025 Official leaderboard
46 GLM-4.1V-9B-Thinking 68% thinking Jul 2025 Official leaderboard
47 InternVL3-14B 67.1% Apr 2025 Official leaderboard
48 MiMo-VL-7B-RL 66.7% May 2025 Official leaderboard
49 Gemini 1.5 Pro (0801) 65.8% Aug 2024 Official leaderboard
50 Ovis2-34B 65.6% Feb 2025 Official leaderboard
51 Gemma 3 27B 64.9% Mar 2025 Official leaderboard
52 MiMo-VL-7B-SFT 64.6% May 2025 Official leaderboard
53 Qwen2-VL-72B-Instruct 64.5% Aug 2024 Official leaderboard
54 Qwen2.5-VL-72B 64.5% Jan 2025 Official leaderboard
55 InternVL2.5-38B 63.9% Dec 2024 Official leaderboard
56 InternVL3-8B 62.7% Apr 2025 Official leaderboard
57 Gemini 1.5 Pro May 2024 62.2% May 2024 Official leaderboard
58 InternVL2-Pro 62% Jul 2024 Official leaderboard
59 Amazon Nova Pro 62% Dec 2024 Official leaderboard
60 TeleMM 61.4% Nov 2024 Official leaderboard
61 Llama 3.2 90B 60.3% Sep 2024 Official leaderboard
62 NVLM-H 1.0 78B 60.2% Sep 2024 Official leaderboard
63 InternVL2.5-26B 60% Dec 2024 Official leaderboard
64 NVLM-D-72B 59.7% Sep 2024 Official leaderboard
65 Ovis2-16B 59.6% Feb 2025 Official leaderboard
66 Gemma 3 12B 59.6% Mar 2025 Official leaderboard
67 Gemini 1.0 Ultra 59.4% Dec 2023 Official leaderboard
68 Claude 3 Opus 59.4% Feb 2024 Official leaderboard
69 GPT-4o mini 59.4% Jul 2024 Official leaderboard
70 Ovis2-8B 59% Feb 2025 Official leaderboard
71 NVILA 58.7% Nov 2024 Official leaderboard
72 Qwen2.5-VL-7B 58.6% Jan 2025 Official leaderboard
73 InternVL2-Llama3-76B 58.3% Jul 2024 Official leaderboard
74 Qianfan-VL-70B 58.3% Sep 2025 Official leaderboard
75 RBDash V1.2 57.7% Aug 2024 Official leaderboard
76 InternVL3-9B 57.7% Apr 2025 Official leaderboard
77 Kimi-VL-A3B 57% Apr 2025 Official leaderboard
78 GPT-4V 56.8% Nov 2023 Official leaderboard
79 LLaVA-OneVision 72B 56.8% Aug 2024 Official leaderboard
80 Reka Core 56.3% Apr 2024 Official leaderboard
81 Gemini 1.5 Flash May 2024 56.1% May 2024 Official leaderboard
82 InternVL2.5-8B 56% Dec 2024 Official leaderboard
83 Eagle2.5-8B 55.8% Apr 2025 Official leaderboard
84 InternVL2-40B 55.2% Jul 2024 Official leaderboard
85 Aria 54.9% Sep 2024 Official leaderboard
86 SenseChat-Vision-0423-Preview 54.6% Apr 2024 Official leaderboard
87 POINTS1.5 53.8% Dec 2024 Official leaderboard
88 Reka Flash 53.3% Feb 2024 Official leaderboard
89 Claude 3 Sonnet 53.1% Feb 2024 Official leaderboard
90 Qwen2.5-VL-3B 53.1% Jan 2025 Official leaderboard
91 EVLM-KTO 53% Dec 2024 Official leaderboard
92 HPT Pro 52% Mar 2024 Official leaderboard
93 InternVL-Chat-V1-2 51.6% Feb 2024 Official leaderboard
94 Qwen-VL-Max 51.4% Jan 2024 Official leaderboard
95 Skywork-VL 51.4% May 2024 Official leaderboard
96 InternVL2-8B 51.2% Jul 2024 Official leaderboard
97 DeepSeek-VL2 51.1% Dec 2024 Official leaderboard
98 SpaRE-7B 51% Apr 2025 Official leaderboard
99 360VL 50.8% May 2024 Official leaderboard
100 MAmmoTH-VL-8B 50.8% Nov 2024 Official leaderboard
101 Llama 3.2 11B 50.7% Sep 2024 Official leaderboard
102 BLIP3-o8B 50.6% May 2025 Official leaderboard
103 Claude 3 Haiku 50.2% Mar 2024 Official leaderboard
104 LLaVA-NeXT-72B 49.9% May 2024 Official leaderboard
105 Ovis1.5-Gemma2-9B 49.8% Jul 2024 Official leaderboard
106 MiniCPM-V-2_6 49.8% Aug 2024 Official leaderboard
107 Cambrian-1-34B 49.7% Jun 2024 Official leaderboard
108 LLaVA-OneVision 7B 48.8% Aug 2024 Official leaderboard
109 Gemma 3 4B 48.8% Mar 2025 Official leaderboard
110 Ovis-Clip-Llama3-8B 48.6% Jun 2024 Official leaderboard
111 InternVL3-2B 48.6% Apr 2025 Official leaderboard
112 Adept Fuyu-Heavy 48.3% Jan 2024 Official leaderboard
113 LLaVA-v1.6-34B 48.1% Jan 2024 Official leaderboard
114 DeepSeek-VL2-Small 48% Dec 2024 Official leaderboard
115 Ovis2-4B 48% Feb 2025 Official leaderboard
116 Gemini 1.0 Pro 47.9% Dec 2023 Official leaderboard
117 Aquila-VL-2B 47.4% Oct 2024 Official leaderboard
118 Qianfan-VL-8B 47.1% Sep 2025 Official leaderboard
119 Idefics3-8B-Llama3 46.6% Aug 2024 Official leaderboard
120 BLIP3-o4B 46.6% May 2025 Official leaderboard
121 Qianfan-VL-3B 46.4% Sep 2025 Official leaderboard
122 Marco-VL-Plus 46.2% Mar 2024 Official leaderboard
123 Yi-VL-34B 45.9% Dec 2023 Official leaderboard
124 MiniCPM-Llama3-V 2.5 45.8% May 2024 Official leaderboard
125 Ovis2-2B 45.3% Feb 2025 Official leaderboard
126 Qwen-VL-Plus 45.2% Jan 2024 Official leaderboard
127 Aya Vision 32B 45.1% Mar 2025 Official leaderboard
128 Mogao-7B 44.2% May 2025 Official leaderboard
129 HPT Air 44% Mar 2024 Official leaderboard
130 InternVL2.5-2B 43.6% Dec 2024 Official leaderboard
131 InternVL3-1B 43.4% Apr 2025 Official leaderboard
132 Bunny-8B 43.3% Jun 2024 Official leaderboard
133 InternLM-XComposer2-VL-7B 43% Jan 2024 Official leaderboard
134 Phi-3.5-vision-instruct 43% Aug 2024 Official leaderboard
135 Flash-VL-2B-Dynamic 42.9% May 2025 Official leaderboard
136 Reka Edge 42.8% Mar 2026 Official leaderboard
137 Cambrian-1-8B 42.7% Jun 2024 Official leaderboard
138 HyViLM 41.8% Dec 2024 Official leaderboard
139 MetaMorph 41.8% Dec 2024 Official leaderboard
140 Bunny-v1.0-4B 41.4% Apr 2024 Official leaderboard
141 Marco-VL 41.2% Jan 2024 Official leaderboard
142 OmniLMM-12B 41.1% Jan 2024 Official leaderboard
143 InternVL2.5-1B 40.9% Dec 2024 Official leaderboard
144 DeepSeek-VL2-Tiny 40.7% Dec 2024 Official leaderboard
145 Cambrian-1-13B 40% Jun 2024 Official leaderboard
146 SpaRE-2B 40% Apr 2025 Official leaderboard
147 Aya Vision 8B 39.9% Mar 2025 Official leaderboard
148 InfiMM-Zephyr-7B 39.4% Jan 2024 Official leaderboard
149 Yi-VL-6B 39.1% Dec 2023 Official leaderboard
150 InternVL-Chat-V1-1 39.1% Jan 2024 Official leaderboard
151 Math-LLaVA-13B 38.3% Jun 2024 Official leaderboard
152 Bunny-v1.0-3B 38.2% Feb 2024 Official leaderboard
153 ILLUME 38.2% Dec 2024 Official leaderboard
154 LLaVA-UHD v2 38.2% Dec 2024 Official leaderboard
155 SVIT 38% Dec 2023 Official leaderboard
156 MiniCPM-V 37.2% Jan 2024 Official leaderboard
157 MiniCPM-V 2.0 37.1% Apr 2024 Official leaderboard
158 LLaVA-v1.5-13B 36.4% Oct 2023 Official leaderboard
159 Emu2-Chat 36.3% Dec 2023 Official leaderboard
160 llava-v1.6-vicuna-13b 36.2% Jan 2024 Official leaderboard
161 Ovis2-1B 36% Feb 2025 Official leaderboard
162 Qwen-VL-Chat 35.9% Aug 2023 Official leaderboard
163 llava-v1.6-vicuna-7b 35.8% Jan 2024 Official leaderboard
164 InstructBLIP FLAN-T5-XXL 35.7% Jun 2023 Official leaderboard
165 BLIP-2 FLAN-T5-XXL 35.4% Feb 2023 Official leaderboard
166 llava-v1.6-mistral-7b 35.3% Jan 2024 Official leaderboard
167 BLIP-2 FLAN-T5-XL 34.4% Feb 2023 Official leaderboard
168 SynerGen-VL 34.2% Dec 2024 Official leaderboard
169 HoVLE 33.7% Dec 2024 Official leaderboard
170 HoVLE (HD) 33.2% Dec 2024 Official leaderboard
171 InstructBLIP FLAN-T5-XL 32.9% May 2023 Official leaderboard
172 SPHINX 32.9% Dec 2023 Official leaderboard
173 Olympus 32.8% Dec 2024 Official leaderboard
174 mPLUG-Owl2 32.7% Nov 2023 Official leaderboard
175 Gemini Nano2 32.6% Dec 2023 Official leaderboard
176 Otter 32.2% Nov 2023 Official leaderboard
177 CogVLM-17B-Chat 32.1% Nov 2023 Official leaderboard
178 Emu3 31.6% Sep 2024 Official leaderboard
179 LLaMA-Adapter2-7B 29.8% Nov 2023 Official leaderboard
180 OpenFlamingo2-9B 28.7% Nov 2023 Official leaderboard
181 Fuyu-8B 27.9% Oct 2023 Official leaderboard
182 Show-o 27.4% Aug 2024 Official leaderboard
183 MiniGPT4-Vicuna-13B 26.8% Nov 2023 Official leaderboard
184 Kosmos-2 24.4% Jun 2023 Official leaderboard

Labs report scores for 97 models in their cards. We plot every source in the charts below.

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
GPT-5
non-thinking thinking
84.2%
Seed1.5-VL
thinking
77.9%
MiMo-VL-7B-RL-2508
non-thinking thinking
70.6%
ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5.1 85.4%
GPT-5 thinking 84.2%
Gemini 2.5 Deep Think 84%
o3 82.9%
o4-mini 81.6%
Claude Opus 4.5 80.7%
dots.vlm1 80.1%
Gemini 2.5 Flash May 2025 79.7%
Gemini 2.5 Pro May 2025 79.6%
Qwen3-VL-235B-A22B 78.7%
o1 78.2%
Grok 3 78%
Seed1.5-VL thinking 77.9%
Claude Sonnet 4.5 77.8%
Intern-S1 77.7%
Claude Opus 4.1 77.1%
Claude Opus 4 76.5%
Llama 4 Behemoth 76.1%
Skywork-R1V3-38B 76%
GLM-4.5V thinking 75.4%
Claude 3.7 Sonnet 75%
Seed1.6-Thinking 74.8%
GPT-4.5 74.4%
Claude Sonnet 4 74.4%
GPT-5 non-thinking 74.4%
Skywork-R1V2-38B 73.6%
Seed1.5-VL 73.6%
Llama 4 Maverick 73.4%
Gemini 2.0 Pro 72.7%
InternVL3-78B 72.2%
Gemini 2.0 Flash Feb 2025 71.7%
GPT-4o Nov 2024 70.7%
MiMo-VL-7B-RL-2508 thinking 70.6%
QVQ-72B-Preview 70.3%
InternVL2_5-78B 70.1%
InternVL3-38B 70.1%
Llama 4 Scout 69.4%
GPT-4o May 2024 69.1%
Skywork-R1V-38B 69%
Claude 3.5 Sonnet 68.3%
Gemini 2.0 Flash-Lite 68%
Amazon Nova Premier 68%
GLM-4.1V-9B-Thinking thinking 68%
InternVL3-14B 67.1%
MiMo-VL-7B-RL 66.7%
Ovis2-34B 65.6%
Gemma 3 27B 64.9%
MiMo-VL-7B-SFT 64.6%
Qwen2-VL-72B-Instruct 64.5%
Qwen2.5-VL-72B 64.5%
InternVL2.5-38B 63.9%
MiMo-VL-7B-RL-2508 non-thinking 63.1%
InternVL3-8B 62.7%
Gemini 1.5 Pro May 2024 62.2%
InternVL2-Pro 62%
Amazon Nova Pro 62%
Llama 3.2 90B 60.3%
InternVL2.5-26B 60%
NVLM-D-72B 59.7%
Ovis2-16B 59.6%
Gemma 3 12B 59.6%
Gemini 1.0 Ultra 59.4%
Claude 3 Opus 59.4%
GPT-4o mini 59.4%
Ovis2-8B 59%
Qwen2.5-VL-7B 58.6%
InternVL2-Llama3-76B 58.3%
Qianfan-VL-70B 58.3%
InternVL3-9B 57.7%
GPT-4V 56.8%
LLaVA-OneVision 72B 56.8%
Reka Core 56.3%
Gemini 1.5 Flash May 2024 56.1%
InternVL2.5-8B 56%
Eagle2.5-8B 55.8%
InternVL2-40B 55.2%
Aria 54.9%
Reka Flash 53.3%
Claude 3 Sonnet 53.1%
Qwen2.5-VL-3B 53.1%
InternVL-Chat-V1-2 51.6%
Qwen-VL-Max 51.4%
InternVL2-8B 51.2%
DeepSeek-VL2 51.1%
MAmmoTH-VL-8B 50.8%
Llama 3.2 11B 50.7%
Claude 3 Haiku 50.2%
LLaVA-NeXT-72B 49.9%
Ovis1.5-Gemma2-9B 49.8%
MiniCPM-V-2_6 49.8%
Cambrian-1-34B 49.7%
LLaVA-OneVision 7B 48.8%
Gemma 3 4B 48.8%
InternVL3-2B 48.6%
LLaVA-v1.6-34B 48.1%
DeepSeek-VL2-Small 48%
Ovis2-4B 48%
Gemini 1.0 Pro 47.9%
Aquila-VL-2B 47.4%
Qianfan-VL-8B 47.1%
Idefics3-8B-Llama3 46.6%
Qianfan-VL-3B 46.4%
Yi-VL-34B 45.9%
MiniCPM-Llama3-V 2.5 45.8%
Ovis2-2B 45.3%
Qwen-VL-Plus 45.2%
Aya Vision 32B 45.1%
InternVL2.5-2B 43.6%
InternVL3-1B 43.4%
InternLM-XComposer2-VL-7B 43%
Phi-3.5-vision-instruct 43%
Reka Edge 42.8%
Cambrian-1-8B 42.7%
Bunny-v1.0-4B 41.4%
OmniLMM-12B 41.1%
InternVL2.5-1B 40.9%
DeepSeek-VL2-Tiny 40.7%
Cambrian-1-13B 40%
Aya Vision 8B 39.9%
Yi-VL-6B 39.1%
InternVL-Chat-V1-1 39.1%
Bunny-v1.0-3B 38.2%
MiniCPM-V 37.2%
MiniCPM-V 2.0 37.1%
LLaVA-v1.5-13B 36.4%
Emu2-Chat 36.3%
llava-v1.6-vicuna-13b 36.2%
Ovis2-1B 36%
Qwen-VL-Chat 35.9%
llava-v1.6-vicuna-7b 35.8%
InstructBLIP FLAN-T5-XXL 35.7%
BLIP-2 FLAN-T5-XXL 35.4%
llava-v1.6-mistral-7b 35.3%
BLIP-2 FLAN-T5-XL 34.4%
InstructBLIP FLAN-T5-XL 32.9%
mPLUG-Owl2 32.7%
CogVLM-17B-Chat 32.1%
Fuyu-8B 27.9%
Kosmos-2 24.4%

10 models have held number one since its release in Nov 2023

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

University of Wisconsin-Madison OpenAI Google DeepMind Anthropic xAI Qwen
Own leaderboard Passed 85% in Nov 2025, by GPT-5.1
GPT-5.1 85.4%
Lab claims Passed 85% in Nov 2025, by GPT-5.1
Qwen 3.6 Plus 86%

The best score passed 85% in Nov 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) Qwen 3.6 Plus GPT-5 o1 GPT-4o Gemini 1.0 Ultra GPT-4V LLaVA-v1.5-13B 0% 20% 40% 60% 80% 100% 2023 2024 2025 2026 Near the ceiling (85%) Qwen 3.6 Plus o1

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
Anthropic
OpenAI
xAI
Moonshot AI
DeepSeek
Qwen
Microsoft
Baidu
Mistral AI
Meta
Amazon
ByteDance Seed
NVIDIA
Z.ai
Shanghai AI Lab
StepFun
Cohere
Liquid AI
Meituan
LG AI Research
Kuaishou
DateLaunch postScoreAs the post writes it
12 Aug 2026 LFM2.5-VL-3B 48.4 MMMU (val)
12 Aug 2026 North Micro Vision Instruct 32.9 MMMU DEV_VAL
25 May 2026 Keye-VL-2.0-30B-A3B 75.3 MMMU
20 May 2026 Command A+ 75.1 MMMU (75.1% in text, 75% in chart)
22 Apr 2026 Qwen3.6 27B 82.9 MMMU
15 Apr 2026 Qwen 3.6 35B-A3B 81.7 MMMU
9 Apr 2026 EXAONE 4.5 33B 78.7 MMMU
2 Apr 2026 Qwen 3.6 Plus 86 MMMU
30 Mar 2026 Qwen3.5-Omni-Plus 80.1 MMMU
4 Mar 2026 Phi-4-Reasoning-Vision-15B 54.3 MMMU_VAL (default mode)
24 Feb 2026 Qwen3.5-122B-A10B 83.9 MMMU
16 Feb 2026 Qwen3.5 397B-A17B 85 MMMU
13 Jan 2026 STEP3-VL-10B 78.1 MMMU (SeRe)
18 Dec 2025 Seed1.8 83.4 MMMU
8 Dec 2025 GLM-4.6V 76 MMMU (Val)
24 Nov 2025 Claude Opus 4.5 80.7 Visual reasoning MMMU (validation)
13 Nov 2025 GPT-5.1 85.4 MMMU
11 Nov 2025 ERNIE-4.5-VL-28B-A3B-Thinking 72.2 MMMU
31 Oct 2025 LongCat-Flash-Omni 70.7 MMMU val
28 Oct 2025 Nemotron Nano 12B v2 VL 68 MMMU* (val, ChatGPT judge)
15 Oct 2025 Claude Haiku 4.5 73.2 Visual reasoning MMMU (validation)
29 Sep 2025 Claude Sonnet 4.5 77.8 Visual reasoning MMMU (validation)
25 Sep 2025 Gemini 2.5 Flash-Lite 72 MMMU single attempt, Thinking
25 Sep 2025 Gemini 2.5 Flash 80.3 MMMU single attempt, model card
23 Sep 2025 Qwen3-VL-235B-A22B 80.6 MMMU_VAL
22 Sep 2025 Qwen3-Omni-30B-A3B 69.1 MMMU_val
25 Aug 2025 InternVL3.5-241B-A28B 77.7 MMMU (val)
12 Aug 2025 LFM2-VL-1.6B 38.4 MMMU (Val)
11 Aug 2025 GLM-4.5V 75.4 MMMU (val)
7 Aug 2025 GPT-5 nano 75.6 MMMU
7 Aug 2025 GPT-5 mini 81.6 MMMU
7 Aug 2025 GPT-5 84.2 MMMU College-level visual problem-solving
5 Aug 2025 Claude Opus 4.1 77.1 Visual reasoning MMMU (validation)
31 Jul 2025 Command A Vision MMMU (chart only)
31 Jul 2025 Step 3 74.2 MMMU
24 Jul 2025 Intern-S1 77.7 MMMU
1 Jul 2025 GLM-4.1V-9B-Thinking 68 MMMU
25 Jun 2025 Seed1.6-Thinking 74.8 MMMU
21 Jun 2025 Kimi-VL-A3B-Thinking-2506 64 MMMU (val, Pass@1)
20 Jun 2025 Mistral Small 3.2 62.5 MMMU
17 Jun 2025 Gemini 2.5 Flash 79.7 MMMU single attempt
17 Jun 2025 Gemini 2.5 Flash-Lite 72.9 MMMU single attempt, Thinking
17 Jun 2025 Gemini 2.5 Pro 82 MMMU, single attempt (pass@1)
3 Jun 2025 Llama-3.1-Nemotron-Nano-VL-8B-V1 48.2 MMMU Val with chatGPT as a judge
22 May 2025 Claude Sonnet 4 74.4 Visual reasoning MMMU (validation)
22 May 2025 Claude Opus 4 76.5 Visual reasoning MMMU (validation)
20 May 2025 BAGEL-7B-MoT 55.3 MMMU
12 May 2025 Seed1.5-VL 77.9 MMMU
7 May 2025 Mistral Medium 3 66.1 MMMU 0-shot
30 Apr 2025 Amazon Nova Premier 68 MMMU
17 Apr 2025 Gemini 2.5 Flash 76.7 MMMU single attempt
16 Apr 2025 o4-mini 81.6 MMMU
16 Apr 2025 o3 82.9 MMMU College-level visual problem-solving
14 Apr 2025 GPT-4.1 nano 55.4 MMMU
14 Apr 2025 GPT-4.1 mini 72.7 MMMU
14 Apr 2025 GPT-4.1 74.8 MMMU
10 Apr 2025 Kimi-VL-A3B-Thinking 61.7 MMMU (val) (Pass@1)
5 Apr 2025 Llama 4 Scout 69.4 MMMU, 0-shot, accuracy
5 Apr 2025 Llama 4 Maverick 73.4 MMMU, 0-shot, accuracy
5 Apr 2025 Llama 4 Behemoth 76.1 MMMU
27 Mar 2025 Qwen2.5-Omni-7B 59.2 MMMUval
25 Mar 2025 Gemini 2.5 Pro 81.7 MMMU, single attempt (pass@1)
17 Mar 2025 Mistral Small 3.1 64 MMMU
16 Mar 2025 ERNIE 4.5 MMMU
12 Mar 2025 Gemma 3 27B 64.9 MMMU (val) (IT); PT MMMU = 56.1
27 Feb 2025 GPT-4.5 74.4 MMMU (multimodal)
26 Feb 2025 Phi-4-multimodal 55.1 MMMU
24 Feb 2025 Claude 3.7 Sonnet 75 Visual reasoning MMMU (validation) (64K extended thinking)
19 Feb 2025 Grok-3 mini 69.4 MMMU [Grok 3 mini, non-reasoning table]
19 Feb 2025 Grok 3 78 MMMU (Grok 3 Think; pass@1 = 76)
5 Feb 2025 Gemini 2.0 Flash-Lite 68 MMMU
5 Feb 2025 Gemini 2.0 Flash 71.7 MMMU
5 Feb 2025 Gemini 2.0 Pro 72.7 MMMU
28 Jan 2025 Qwen2.5-VL-72B 70.2 MMMUval
27 Jan 2025 Janus-Pro-7B 41 MMMU
21 Jan 2025 Gemini 2.0 Flash Thinking 75.4 MMMU (Multimodal reasoning)
20 Jan 2025 Kimi k1.5 70 MMMU-Val (Pass@1), long-CoT
17 Dec 2024 o1 77.3 MMMU (pass @1)
11 Dec 2024 Gemini 2.0 Flash 70.7 MMMU
22 Oct 2024 Claude 3.5 Sonnet (October 2024) 70.4 MMMU (validation)
12 Sep 2024 o1-preview MMMU (val) pass@1
13 Aug 2024 Grok-2 66.1 MMMU (0-shot CoT)
18 Jul 2024 GPT-4o mini 59.4 MMMU
20 Jun 2024 Claude 3.5 Sonnet 68.3 MMMU (validation)
14 May 2024 Gemini 1.5 Pro 62.2 MMMU (val), 0-shot
13 May 2024 GPT-4o 69.1 MMMU (%) (val)
4 Mar 2024 Claude 3 Opus 59.4 MMMU (val) Overall
15 Feb 2024 Gemini 1.5 Pro 58.5 MMMU (val), 0-shot
6 Dec 2023 Gemini 1.0 Ultra 62.4 MMMU (val), Maj1@32 (pass@1 = 59.4%)

51 system cards report it, from 18 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
9 Jun 2026 Keye-VL-2.0-30B-A3B technical report 75.3 MMMU (Figure 5, printed value) Capabilities
27 Apr 2026 Nemotron 3 Nano Omni 30B-A3B technical report 70.8 MMMU (val), reasoning on, 55.2 reasoning off (Table 7) Capabilities
17 Apr 2026 Qwen3.5-Omni-Plus technical report 80.1 MMMU (Table 6) Capabilities
9 Apr 2026 EXAONE 4.5 33B technical report 78.7 MMMU (Table 2, Reasoning mode) Capabilities
21 Mar 2026 Seed1.8 model card 83.4 MMMU (Table 2) Capabilities
4 Mar 2026 Phi-4-Reasoning-Vision-15B technical report 54.3 MMMU VAL, default mode, 52 force nothink, 55 force thinking (Tables 5 and 6) Capabilities
14 Feb 2026 Seed2.0 Pro model card 85.4 MMMU (Table 8) Capabilities
30 Jan 2026 Kimi K2.5 technical report 84.3 MMMU (val) (Table 4) Capabilities
14 Jan 2026 STEP3-VL-10B technical report 78.1 MMMU, SeRe (Tables 1 and 3). PaCoRe 80.11 Capabilities
1 Jan 2026 GLM-4.6V technical report 76 MMMU (Val), GLM-4.6V thinking (Table 2) Capabilities
28 Nov 2025 LFM2-VL-1.6B technical report 39.7 MMMU, LFM2-VL-1.6B (Table 8) Capabilities
26 Nov 2025 Qwen3-VL-235B-A22B technical report 80.6 MMMU (Table 2) Capabilities
24 Nov 2025 Claude Opus 4.5 system card 80.7 MMMU (validation), 64k thinking, avg of 5 trials (Table 2.3.A; 80.72% in 2.20) Capabilities
6 Nov 2025 Nemotron Nano 12B v2 VL technical report 67.8 MMMU (val), reasoning on (Table 3). Reasoning off 55.3 Capabilities
31 Oct 2025 LongCat-Flash-Omni technical report 70.7 MMMU val (Table 5) Capabilities
26 Sep 2025 Gemini 2.5 Flash model card 80.3 Visual reasoning MMMU, single attempt (pass@1) (Table, p.5) Capabilities
25 Sep 2025 gemini-robotics-er-1.5-preview technical report 80.7 MMMU (Table 20, thinking on). Thinking off 79.3 Capabilities
22 Sep 2025 Qwen3-Omni-30B-A3B technical report 69.1 MMMU val, Instruct (Table 9). Thinking 75.6 (Table 10) Capabilities
25 Aug 2025 InternVL3.5-241B-A28B technical report 77.7 MMMU (val) (Table 2, Table 3). With parallel thinking 78.7 Capabilities
21 Aug 2025 Intern-S1 technical report 77.7 MMMU (Table 2) Capabilities
13 Aug 2025 GLM-4.5V technical report 75.4 MMMU (Val), GLM-4.5V thinking (Table 2) Capabilities
7 Jul 2025 Gemini 2.5 Pro technical report 82 MMMU, single attempt (Table 3) Capabilities
7 Jul 2025 Gemini 2.5 Flash technical report 79.7 MMMU, single attempt (Table 3) Capabilities
1 Jul 2025 GLM-4.1V-9B-Thinking technical report 68 MMMU (Val), GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities
23 Jun 2025 Kimi-VL-A3B-Thinking-2506 technical report 64 MMMU (val) (Pass@1) (Table 4) Capabilities
12 Jun 2025 Magistral Small 1.0 technical report 66 MMMU, Magistral Small bar, printed label (Figure 10) Capabilities
12 Jun 2025 Magistral Medium 1.0 technical report 70 MMMU, printed bar label (Figure 10) Capabilities
20 May 2025 BAGEL-7B-MoT technical report 55.3 MMMU (Table 4) Capabilities
11 May 2025 Seed1.5-VL technical report 77.9 MMMU, thinking (Table 6). Non-thinking 73.6 Capabilities
30 Apr 2025 Amazon Nova Premier technical report 68 MMMU, CoT (Table 1) Capabilities
29 Apr 2025 Gemini 2.5 Flash model card 76.7 MMMU, single attempt (pass@1) (Table, p.4) Capabilities
28 Apr 2025 Gemini 2.5 Pro model card 81.7 MMMU, single attempt pass@1 (Table, p.4, column Gemini 2.5 Pro Experimental 03-25) Capabilities
15 Apr 2025 Gemini 2.0 Flash model card 71.7 MMMU (Table, p.3, column Gemini 2.0 Flash (GA)) Capabilities
10 Apr 2025 Kimi-VL-A3B-Thinking technical report 61.7 MMMU (val) (Pass@1) (Table 4). Figure 13 shows 49.2 at 1k rising to 61.7 at 16k max thinking tokens Capabilities
10 Apr 2025 Gemini 2.0 Flash-Lite model card 68 MMMU (Table, p.3) Capabilities
5 Apr 2025 Llama 4 Scout model card 69.4 MMMU, 0-shot, accuracy, instruction tuned (Instruction tuned models table) Capabilities
5 Apr 2025 Llama 4 Maverick model card 73.4 MMMU, 0-shot, accuracy, instruction tuned (Instruction tuned models table) Capabilities
26 Mar 2025 Qwen2.5-Omni-7B technical report 59.2 MMMU val (Table 5) Capabilities
25 Mar 2025 Gemma 3 4B technical report 48.8 MMMU (val), IT (Table 6 and Table 16) Capabilities
25 Mar 2025 Gemma 3 27B technical report 64.9 MMMU (val), IT (Table 6 and Table 16) Capabilities
25 Mar 2025 Gemma 3 12B technical report 59.6 MMMU (val), IT (Table 6 and Table 16) Capabilities
3 Mar 2025 Phi-4-multimodal technical report 55.1 MMMU (val) (Table 1) Capabilities
19 Feb 2025 Qwen2.5-VL-72B technical report 70.2 MMMU val (Table 3) Capabilities
27 Jan 2025 Janus-Pro-7B technical report 41 MMMU (Table 3) Capabilities
20 Jan 2025 Kimi k1.5 technical report 70 MMMU-Val (Pass@1), long-CoT (Table 2). Short-CoT 68.0 (Table 3) Capabilities
22 Oct 2024 Claude 3.5 Sonnet (October 2024) model card 70.4 MMMU (validation), 0-shot CoT (Table 5) Capabilities
30 Aug 2024 Phi-3.5-vision-instruct technical report 43 MMMU (val), 0-shot (Table 5) Capabilities
20 Jun 2024 Claude 3.5 Sonnet model card 68.3 MMMU (validation), 0-shot CoT (Table 2) Capabilities
4 Mar 2024 Claude 3 Sonnet model card 53.1 MMMU (val), Overall, 0-shot. Per-discipline values also given (Table 3) Capabilities
4 Mar 2024 Claude 3 Opus model card 59.4 MMMU (val), Overall, 0-shot. Per-discipline values also given (Table 3) Capabilities
4 Mar 2024 Claude 3 Haiku model card 50.2 MMMU (val), Overall, 0-shot. Per-discipline values also given (Table 3) Capabilities

Versions

Replaced by MMMU-Pro

More in Vision

All 234 in Vision
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%