Benchmark Almanac

CVBench

Seed1.8 88% leads since Dec 2025 Fading

Models answer 2,638 manually checked 2D and 3D vision questions built from ADE20K, COCO, and Omni3D. Scoring is accuracy.

Made by New York University (Cambrian-1 team) Released Jun 2024 Spatial reasoning and counting All tasks public Last in a launch post Sep 2026 , dropped after it saturated Last model added Sep 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
cambrian-mllm/cambrian 2,013 stars Paper Dataset cambrian-mllm.github.io
11
Models scored in lab cards
6
Launch posts list it, from 5 labs
9
System cards report it, from 6 labs

Seed1.8 leads with 88% among models released by Sep 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in Sep 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 88.0 Seed1.8
  2. 87.6 GLM-4.6V
  3. 87.3 GLM-4.5V
  4. 87.2 ERNIE 5.0
  5. 85.0 GLM-4.1V-9B-Thinking
  6. 84.3 gemini-robotics-er-1.5-preview
  7. 84.0 Nemotron 3 Nano Omni 30B-A3B
  8. 83.8 ERNIE-4.5-VL-28B-A3B-Thinking
  9. 83.5 STEP3-VL-10B
  10. 78.3 Nemotron Nano 12B v2 VL
  11. 77.9 DeepSeek V4.1 Flash
  1. 1 Seed1.8 88%
  2. 2 GLM-4.6V 87.6%
  3. 3 GLM-4.5V 87.3%
  4. 4 ERNIE 5.0 87.2%
  5. 5 GLM-4.1V-9B-Thinking 85%
  6. 6 gemini-robotics-er-1.5-preview 84.3%
  7. 7 Nemotron 3 Nano Omni 30B-A3B 84%
  8. 8 ERNIE-4.5-VL-28B-A3B-Thinking 83.8%
  9. 9 STEP3-VL-10B 83.5%
  10. 10 Nemotron Nano 12B v2 VL 78.3%
  11. 11 DeepSeek V4.1 Flash 77.9%
#ModelScoreSettingDateSource
1 Seed1.8 88% Dec 2025 Model card
2 GLM-4.6V 87.6% Jan 2026 Model card
3 GLM-4.5V 87.3% Aug 2025 Model card
4 ERNIE 5.0 87.2% Feb 2026 Model card
5 GLM-4.1V-9B-Thinking 85% Jul 2025 Model card
6 gemini-robotics-er-1.5-preview 84.3% Sep 2025 Model card
7 Nemotron 3 Nano Omni 30B-A3B 84% Apr 2026 Model card
8 ERNIE-4.5-VL-28B-A3B-Thinking 83.8% Nov 2025 Model card
9 STEP3-VL-10B 83.5% Jan 2026 Model card
10 Nemotron Nano 12B v2 VL 78.3% Nov 2025 Model card
11 DeepSeek V4.1 Flash 77.9% Sep 2026 Model card

3 models have held number one since its release in Jun 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Z.ai ByteDance Seed
Lab claims Passed 85% in Jul 2025, by GLM-4.1V-9B-Thinking
Seed1.8 88%

The best score passed 85% in Jul 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Seed1.8 GLM-4.5V 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Seed1.8

Labs headline it less often now, last in Sep 2026

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Z.ai
Baidu
ByteDance Seed
NVIDIA
DeepSeek
DateLaunch postScoreAs the post writes it
10 Sep 2026 DeepSeek V4.1 Flash 77.9 CVBench (EM)
28 Apr 2026 Nemotron 3 Nano Omni 30B-A3B 84 CVBench2D
6 Feb 2026 ERNIE 5.0 87.2 CV-Bench
18 Dec 2025 Seed1.8 88 CV-Bench
11 Nov 2025 ERNIE-4.5-VL-28B-A3B-Thinking 83.8 CV-Bench
11 Aug 2025 GLM-4.5V 87.3 CV-Bench

9 system cards report it, from 6 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
27 Apr 2026 Nemotron 3 Nano Omni 30B-A3B technical report 84 CV-Bench, reasoning on, 84.2 reasoning off (Table 7) Capabilities
21 Mar 2026 Seed1.8 model card 88 CV-Bench (Table 2) Capabilities
4 Feb 2026 ERNIE 5.0 technical report 87.2 CV-Bench (Table 3) Capabilities
14 Jan 2026 STEP3-VL-10B technical report 83.5 CVBench, SeRe (Tables 1 and 3). PaCoRe 85.92 Capabilities
1 Jan 2026 GLM-4.6V technical report 87.6 CV-Bench, GLM-4.6V thinking (Table 2) Capabilities
6 Nov 2025 Nemotron Nano 12B v2 VL technical report 78.3 CV-Bench, reasoning on (Table 3). Reasoning off 81.0 Capabilities
25 Sep 2025 gemini-robotics-er-1.5-preview technical report 84.3 CV-Bench (Table 19, thinking on). Thinking off 83.6 Capabilities
13 Aug 2025 GLM-4.5V technical report 87.3 CV-Bench, GLM-4.5V thinking (Table 2) Capabilities
1 Jul 2025 GLM-4.1V-9B-Thinking technical report 85 CV-Bench, GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%