BabyVision
Model solves 388 perception-heavy visual puzzles (discrimination, tracking, spatial, patterns) that young children solve. Accuracy versus 94% human baseline.
Moonshot AI is a listed collaborator.
In the authors’ words
While humans develop core visual skills long before acquiring language, contemporary Multimodal LLMs (MLLMs) still rely heavily on linguistic priors to compensate for their fragile visual understanding. We uncovered a crucial fact: state-of-the-art MLLMs consistently fail on basic visual tasks that humans, even 3-year-olds, can solve effortlessly. To systematically investigate this gap, we introduce BabyVision, a benchmark designed to assess core visual abilities independent of linguistic knowledge for MLLMs. BabyVision spans a wide range of tasks, with 388 items divided into 22 subclasses across four key categories. Empirical results and human evaluation reveal that leading MLLMs perform significantly below human baselines. Gemini3-Pro-Preview scores 49.7, lagging behind 6-year-old humans and falling well behind the average adult score of 94.1. These results show despite excelling in knowledge-heavy evaluations, current MLLMs still lack fundamental visual primitives. Progress in BabyVision represents a step toward human-level visual perception and reasoning capabilities. We also explore solving visual reasoning with generation models by proposing BabyVision-Gen and automatic evaluation toolkit. Our code and benchmark data are released at https://github.com/UniPat-AI/BabyVision for reproduction.
- Level
- Everyday Most adults, without training
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Short answer A number, word or short string matched to a key.
Qwen 3.8 Max leads with 91.3% among models released by Aug 2026
Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.
The newest model with a score here came out in Aug 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 11 more
- 91.3
Qwen 3.8 Max
- 90.5 Claude Fable 5
- 88.9 GPT-5.6 Sol
- 85.7 Kimi K3
- 83.6 GPT-5.5
- 81.2 Claude Opus 4.8
- 76.3
Muse Spark 1.1
- 73.7
Seed2.1 Pro
- 70.4
Qwen3.7-Plus
- 68.5 Kimi K2.6
- 60.6
Seed2.0 Pro
- 51.6
Gemini 3.1 Pro
- 49.7
Gemini 3 Pro
- 49.7 GPT-5.4
- 43.3
Qwen3.5 397B-A17B
- 36.5 Kimi K2.5
- 34.4 GPT-5.2
- 30.2
Seed1.8
- 22.2
Qwen3-VL-235B-A22B
- 19.2
InternVL3.5-241B-A28B
- 19.2
Qwen3-VL-Plus
- 17.6 GLM-4.6V
- 16.2 Grok 4
- 15.1 MiMo-VL-7B-RL
- 14.8 Claude Opus 4.6
- 14.7
Step 3
- 14.2 Claude Opus 4.5
- 12.4 KimiVL-A3B
- 1
Qwen 3.8 Max 91.3%
- 2 Claude Fable 5 max 90.5%
- 3 GPT-5.6 Sol max 88.9%
- 4 Kimi K3 85.7%
- 5 GPT-5.5 xhigh 83.6%
- 6 Claude Opus 4.8 max 81.2%
- 7
Muse Spark 1.1 76.3%
- 8
Seed2.1 Pro 73.7%
- 9
Qwen3.7-Plus 70.4%
- 10 Kimi K2.6 68.5%
- 11
Seed2.0 Pro high 60.6%
- 12
Gemini 3.1 Pro 51.6%
- 13
Gemini 3 Pro 49.7%
- 14 GPT-5.4 xhigh 49.7%
- 15
Qwen3.5 397B-A17B 43.3%
- 16 Kimi K2.5 36.5%
- 17 GPT-5.2 34.4%
- 18
Seed1.8 30.2%
- 19
Qwen3-VL-235B-A22B 22.2%
- 20
InternVL3.5-241B-A28B 19.2%
- 21
Qwen3-VL-Plus 19.2%
- 22 GLM-4.6V 17.6%
- 23 Grok 4 16.2%
- 24 MiMo-VL-7B-RL 15.1%
- 25 Claude Opus 4.6 14.8%
- 26
Step 3 14.7%
- 27 Claude Opus 4.5 14.2%
- 28 KimiVL-A3B 12.4%
| # | Entry | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 91.3% | tools | Aug 2026 | Official leaderboard |
| 2 | Claude Fable 5 | 90.5% | max, tools | Jun 2026 | Official leaderboard |
| 3 | GPT-5.6 Sol | 88.9% | max, tools | Jul 2026 | Official leaderboard |
| 4 | Kimi K3 | 85.7% | Python tools | Jul 2026 | Official leaderboard |
| 5 | GPT-5.5 | 83.6% | xhigh, tools | Apr 2026 | Official leaderboard |
| 6 | Claude Opus 4.8 | 81.2% | max, tools | May 2026 | Official leaderboard |
| 7 | | 76.3% | tools | Jul 2026 | Official leaderboard |
| 8 | | 73.7% | Jun 2026 | Official leaderboard | |
| 9 | | 70.4% | tools | Jun 2026 | Official leaderboard |
| 10 | Kimi K2.6 | 68.5% | tools | Apr 2026 | Official leaderboard |
| 11 | | 60.6% | high | Feb 2026 | Official leaderboard |
| 12 | | 51.6% | Feb 2026 | Official leaderboard | |
| 13 | | 49.7% | Nov 2025 | Official leaderboard | |
| 14 | GPT-5.4 | 49.7% | xhigh | Mar 2026 | Official leaderboard |
| 15 | | 43.3% | Feb 2026 | Official leaderboard | |
| 16 | Kimi K2.5 | 36.5% | Jan 2026 | Official leaderboard | |
| 17 | GPT-5.2 | 34.4% | Dec 2025 | Official leaderboard | |
| 18 | | 30.2% | Dec 2025 | Official leaderboard | |
| 19 | | 22.2% | Sep 2025 | Official leaderboard | |
| 20 | | 19.2% | Aug 2025 | Official leaderboard | |
| 21 | | 19.2% | Sep 2025 | Official leaderboard | |
| 22 | GLM-4.6V | 17.6% | Dec 2025 | Official leaderboard | |
| 23 | Grok 4 | 16.2% | Jul 2025 | Official leaderboard | |
| 24 | MiMo-VL-7B-RL | 15.1% | May 2025 | Official leaderboard | |
| 25 | Claude Opus 4.6 | 14.8% | Feb 2026 | Official leaderboard | |
| 26 | | 14.7% | Jul 2025 | Official leaderboard | |
| 27 | Claude Opus 4.5 | 14.2% | Nov 2025 | Official leaderboard | |
| 28 | KimiVL-A3B | 12.4% | Apr 2025 | Official leaderboard |
Labs report scores for 20 models in their cards. We plot every source in the charts below.
6 models have held number one since its release in Jan 2026
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
The best score passed 85% in Jun 2026
All the scores we have, one dot each, so a model tested at several settings appears more than once.
8 labs put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 24 Oct 2026 | Kimi K3.1 Pro | 95.4 | BabyVision w/ python |
| 19 Oct 2026 | 91 | BabyVision (with tools) | |
| 17 Oct 2026 | 92.5 | BabyVision (w/tools) | |
| 15 Oct 2026 | 80.2 | BabyVision (w/ tools) | |
| 15 Oct 2026 | 29.6 | BabyVision | |
| 10 Sep 2026 | 89.6 | BabyVision w/ tools (Pass@1) | |
| 26 Aug 2026 | GLM-5.3-Flash | 53.4 | BabyVision |
| 14 Aug 2026 | 85.6 | General visual reasoning BabyVision Without CI 65.7 With CI 85.6 | |
| 3 Aug 2026 | 91.3 | BabyVision (without CI / with CI) 82.0 / 91.3 | |
| 16 Jul 2026 | Kimi K3 | 85.7 | BabyVision w/ python |
| 9 Jul 2026 | 76.3 | BabyVision (Visual reasoning) | |
| 23 Jun 2026 | 73.7 | BabyVision | |
| 1 Jun 2026 | 70.4 | BabyVision 70.4 / 64.7 (with CI / without CI) | |
| 28 Apr 2026 | 64.7 | BabyVision | |
| 20 Apr 2026 | Kimi K2.6 | 39.8 | BabyVision |
| 9 Apr 2026 | 18.8 | BabyVision | |
| 24 Feb 2026 | 40.2 | BabyVision 40.2 / 34.5 (with CI / without CI) | |
| 16 Feb 2026 | 52.3 | BabyVision (52.3/43.3) | |
| 14 Feb 2026 | 60.6 | BabyVision |
8 system cards report it, from 6 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 15 Oct 2026 | | 29.6 | BabyVision (Table 10) | Capabilities |
| 10 Sep 2026 | | 89.6 | BabyVision w/ tools (Pass@1) (Table 3) | Capabilities |
| 27 Jul 2026 | Kimi K3 technical report | 85.7 | BabyVision w/ Python, the only setting reported (Table 2) | Capabilities |
| 9 Jul 2026 | | 76.3 | BabyVision (w/ tools) (Figure 44) | Capabilities |
| 23 Jun 2026 | | 73.7 | BabyVision (Table 9) | Capabilities |
| 9 Apr 2026 | | 18.8 | BabyVision (Table 2, Reasoning mode) | Capabilities |
| 14 Feb 2026 | | 60.6 | BabyVision (Table 8) | Capabilities |
| 30 Jan 2026 | Kimi K2.5 technical report | 36.5 | BabyVision (Table 4) | Capabilities |
More in Vision
- MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
- MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
- CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
- MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy.
Qwen 3.8 Max 97.7%
- ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance.
Command A Vision 90.9%
- DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers.
Seed1.5-VL 96.9%