Benchmark Almanac

BabyVision

Qwen 3.8 Max 91.3% leads since Aug 2026 In use

Model solves 388 perception-heavy visual puzzles (discrimination, tracking, spatial, patterns) that young children solve. Accuracy versus 94% human baseline.

Moonshot AI is a listed collaborator.

In the authors’ words

While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguistic priors to compensate for their fragile visual understanding. We uncovered a crucial fact: state-of-the-art MLLMs consistently fail on basic visual tasks that humans, even 3-year-olds, can solve effortlessly. To systematically investigate this gap, we introduce BabyVision, a benchmark designed to assess core visual abilities independent of linguistic knowledge for MLLMs. BabyVision spans a wide range of tasks, with 388 items divided into 22 subclasses across four key categories. Empirical results and human evaluation reveal that leading MLLMs perform significantly below human baselines. Gemini3-Pro-Preview scores 49.7, lagging behind 6-year-old humans and falling well behind the average adult score of 94.1. These results show despite excelling in knowledge-heavy evaluations, current MLLMs still lack fundamental visual primitives. Progress in BabyVision represents a step toward human-level visual perception and reasoning capabilities. We also explore solving visual reasoning with generation models by proposing BabyVision-Gen and automatic evaluation toolkit. Our code and benchmark data are released at https://github.com/UniPat-AI/BabyVision for reproduction.

Abstract of BabyVision: Visual Reasoning Beyond Language (arXiv:2601.06521, CC0)
Made by UniPat AI + xbench Released Jan 2026 Spatial reasoning and counting All tasks public Last in a launch post Oct 2026 Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
28
Entries on its own leaderboard, 27 models we track
19
Launch posts list it, from 9 labs
8
System cards report it, from 6 labs

Qwen 3.8 Max leads with 91.3% among models released by Aug 2026

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

The newest model with a score here came out in Aug 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Qwen 3.8 Max 91.3%
  2. 2 Claude Fable 5 max 90.5%
  3. 3 GPT-5.6 Sol max 88.9%
  4. 4 Kimi K3 85.7%
  5. 5 GPT-5.5 xhigh 83.6%
  6. 6 Claude Opus 4.8 max 81.2%
  7. 7 Muse Spark 1.1 76.3%
  8. 8 Seed2.1 Pro 73.7%
  9. 9 Qwen3.7-Plus 70.4%
  10. 10 Kimi K2.6 68.5%
  11. 11 Seed2.0 Pro high 60.6%
  12. 12 Gemini 3.1 Pro 51.6%
  13. 13 Gemini 3 Pro 49.7%
  14. 14 GPT-5.4 xhigh 49.7%
  15. 15 Qwen3.5 397B-A17B 43.3%
  16. 16 Kimi K2.5 36.5%
  17. 17 GPT-5.2 34.4%
  18. 18 Seed1.8 30.2%
  19. 19 Qwen3-VL-235B-A22B 22.2%
  20. 20 InternVL3.5-241B-A28B 19.2%
  21. 21 Qwen3-VL-Plus 19.2%
  22. 22 GLM-4.6V 17.6%
  23. 23 Grok 4 16.2%
  24. 24 MiMo-VL-7B-RL 15.1%
  25. 25 Claude Opus 4.6 14.8%
  26. 26 Step 3 14.7%
  27. 27 Claude Opus 4.5 14.2%
  28. 28 KimiVL-A3B 12.4%
#EntryScoreSettingDateSource
1 Qwen 3.8 Max 91.3% tools Aug 2026 Official leaderboard
2 Claude Fable 5 90.5% max, tools Jun 2026 Official leaderboard
3 GPT-5.6 Sol 88.9% max, tools Jul 2026 Official leaderboard
4 Kimi K3 85.7% Python tools Jul 2026 Official leaderboard
5 GPT-5.5 83.6% xhigh, tools Apr 2026 Official leaderboard
6 Claude Opus 4.8 81.2% max, tools May 2026 Official leaderboard
7 Muse Spark 1.1 76.3% tools Jul 2026 Official leaderboard
8 Seed2.1 Pro 73.7% Jun 2026 Official leaderboard
9 Qwen3.7-Plus 70.4% tools Jun 2026 Official leaderboard
10 Kimi K2.6 68.5% tools Apr 2026 Official leaderboard
11 Seed2.0 Pro 60.6% high Feb 2026 Official leaderboard
12 Gemini 3.1 Pro 51.6% Feb 2026 Official leaderboard
13 Gemini 3 Pro 49.7% Nov 2025 Official leaderboard
14 GPT-5.4 49.7% xhigh Mar 2026 Official leaderboard
15 Qwen3.5 397B-A17B 43.3% Feb 2026 Official leaderboard
16 Kimi K2.5 36.5% Jan 2026 Official leaderboard
17 GPT-5.2 34.4% Dec 2025 Official leaderboard
18 Seed1.8 30.2% Dec 2025 Official leaderboard
19 Qwen3-VL-235B-A22B 22.2% Sep 2025 Official leaderboard
20 InternVL3.5-241B-A28B 19.2% Aug 2025 Official leaderboard
21 Qwen3-VL-Plus 19.2% Sep 2025 Official leaderboard
22 GLM-4.6V 17.6% Dec 2025 Official leaderboard
23 Grok 4 16.2% Jul 2025 Official leaderboard
24 MiMo-VL-7B-RL 15.1% May 2025 Official leaderboard
25 Claude Opus 4.6 14.8% Feb 2026 Official leaderboard
26 Step 3 14.7% Jul 2025 Official leaderboard
27 Claude Opus 4.5 14.2% Nov 2025 Official leaderboard
28 KimiVL-A3B 12.4% Apr 2025 Official leaderboard

Labs report scores for 20 models in their cards. We plot every source in the charts below.

6 models have held number one since its release in Jan 2026

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.

Google DeepMind ByteDance Seed Moonshot AI OpenAI Anthropic Qwen Meta DeepSeek
Own leaderboard Passed 85% in Jun 2026, by Claude Fable 5
Qwen 3.8 Max 91.3%
Lab claims Passed 85% in Jul 2026, by Kimi K3

The best score passed 85% in Jun 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Kimi K3.1 Pro Claude Fable 5 GPT-5.5 Kimi K2.6 Seed2.0 Pro Gemini 3 Pro Qwen3-VL-235B-A22B 0% 20% 40% 60% 80% 100% 2025 2026 2027 Near the ceiling (85%) Kimi K3.1 Pro GPT-5.5

8 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

ByteDance Seed
Qwen
LG AI Research
Moonshot AI
Meta
Z.ai
DeepSeek
NVIDIA
MiniMax
DateLaunch postScoreAs the post writes it
24 Oct 2026 Kimi K3.1 Pro 95.4 BabyVision w/ python
19 Oct 2026 MiniMax-M3.1-Flash 91 BabyVision (with tools)
17 Oct 2026 DeepSeek V4.1 Flash 1017 92.5 BabyVision (w/tools)
15 Oct 2026 Muse Spark Open 80.2 BabyVision (w/ tools)
15 Oct 2026 Nemotron 3.5 Super VL 29.6 BabyVision
10 Sep 2026 DeepSeek V4.1 Flash 89.6 BabyVision w/ tools (Pass@1)
26 Aug 2026 GLM-5.3-Flash 53.4 BabyVision
14 Aug 2026 Qwen 3.8 27B 85.6 General visual reasoning BabyVision Without CI 65.7 With CI 85.6
3 Aug 2026 Qwen 3.8 Max 91.3 BabyVision (without CI / with CI) 82.0 / 91.3
16 Jul 2026 Kimi K3 85.7 BabyVision w/ python
9 Jul 2026 Muse Spark 1.1 76.3 BabyVision (Visual reasoning)
23 Jun 2026 Seed2.1 Pro 73.7 BabyVision
1 Jun 2026 Qwen3.7-Plus 70.4 BabyVision 70.4 / 64.7 (with CI / without CI)
28 Apr 2026 Seed2.0 Lite (0428) 64.7 BabyVision
20 Apr 2026 Kimi K2.6 39.8 BabyVision
9 Apr 2026 EXAONE 4.5 33B 18.8 BabyVision
24 Feb 2026 Qwen3.5-122B-A10B 40.2 BabyVision 40.2 / 34.5 (with CI / without CI)
16 Feb 2026 Qwen3.5 397B-A17B 52.3 BabyVision (52.3/43.3)
14 Feb 2026 Seed2.0 Pro 60.6 BabyVision

8 system cards report it, from 6 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
15 Oct 2026 Nemotron 3.5 Super VL technical report 29.6 BabyVision (Table 10) Capabilities
10 Sep 2026 DeepSeek V4.1 Flash technical report 89.6 BabyVision w/ tools (Pass@1) (Table 3) Capabilities
27 Jul 2026 Kimi K3 technical report 85.7 BabyVision w/ Python, the only setting reported (Table 2) Capabilities
9 Jul 2026 Muse Spark 1.1 system card 76.3 BabyVision (w/ tools) (Figure 44) Capabilities
23 Jun 2026 Seed2.1 Pro model card 73.7 BabyVision (Table 9) Capabilities
9 Apr 2026 EXAONE 4.5 33B technical report 18.8 BabyVision (Table 2, Reasoning mode) Capabilities
14 Feb 2026 Seed2.0 Pro model card 60.6 BabyVision (Table 8) Capabilities
30 Jan 2026 Kimi K2.5 technical report 36.5 BabyVision (Table 4) Capabilities

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%