Benchmark Almanac

MME-CC

Seed2.0 Pro 57% leads since Feb 2026 Fading

Models answer 1,173 expert-annotated vision-grounded questions in 11 task types grouped as spatial, geometric and visual knowledge reasoning.

At release Gemini 2.5 Pro led 16 models with 42.66. The scoring field is a guess from the abstract.

In the authors’ words

As reasoning models scale rapidly, the essential role of multimodality in human cognition has come into sharp relief, driving a growing need to probe vision-centric cognitive behaviors. Yet, existing multimodal benchmarks either overemphasize textual reasoning or fall short of systematically capturing vision-centric cognitive behaviors, leaving the cognitive capacity of MLLMs insufficiently assessed. To address this limitation, we introduce MME-CC (Multi-Modal Evaluation benchmark of Cognitive Capacity), a vision-grounded benchmark that organizes 11 representative reasoning tasks into three fundamental categories of visual information: spatial, geometric, and knowledge-based reasoning, and provides fine-grained analyses of MLLMs' cognitive capacity across these dimensions. Based on MME-CC, we conduct extensive experiments over 16 representative MLLMs. Our study reveals that closed-source models currently lead overall (e.g., 42.66 for Gemini-2.5-Pro vs. 30.45 for GLM-4.5V), while spatial and geometric reasoning remain broadly weak (less than or equal to 30%). We further identify common error patterns, including orientation mistakes, fragile cross-view identity persistence, and poor adherence to counterfactual instructions, and observe that Chain-of-Thought typically follows a three-stage process (extract -> reason -> verify) with heavy reliance on visual extraction. We hope this work catalyzes a shift toward treating the cognitive capacity of MLLMs as central to both evaluation and model design.

Abstract of MME-CC: A Challenging Multi-Modal Evaluation Benchmark of Cognitive Capacity (arXiv:2511.03146, CC0)
Made by ByteDance Seed and collaborators (Zhang et al.) Released Nov 2025 Spatial reasoning and counting Last in a launch post Dec 2025 , dropped before it saturated Last model added Mar 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
2
Models scored in lab cards
1
Launch post lists it, from 1 lab
2
System cards report it, from 1 lab

Seed2.0 Pro leads with 57%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

  1. 1 Seed2.0 Pro 57%
  2. 2 Seed1.8 43.4%
#ModelScoreSettingDateSource
1 Seed2.0 Pro 57% Feb 2026 Model card
2 Seed1.8 43.4% Dec 2025 Model card

2 models have held number one since its release in Nov 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

ByteDance Seed
Lab claims Not tested since Mar 2026

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Seed2.0 Pro Seed1.8 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Seed2.0 Pro Seed1.8

Labs headline it less often now, last in Dec 2025

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

ByteDance Seed
DateLaunch postScoreAs the post writes it
18 Dec 2025 Seed1.8 43.4 MME-CC

2 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
21 Mar 2026 Seed1.8 model card 43.4 MME-CC (Table 2) Capabilities
14 Feb 2026 Seed2.0 Pro model card 57 MME-CC (Table 8) Capabilities

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%