Benchmark Almanac

TreeBench

Seed2.1 Pro 71.1% leads since Jun 2026 Fading

Models answer 405 visual questions about small targets in dense SA-1B images and give bounding boxes as evidence. Scored by answer accuracy, with box overlap as a second measure.

At release no model reached 60% accuracy, and OpenAI o3 scored 54.87.

In the authors’ words

Models like OpenAI-o3 pioneer visual grounded reasoning by dynamically referencing visual regions, just like human "thinking with images". However, no benchmark exists to evaluate these capabilities holistically. To bridge this gap, we propose TreeBench (Traceable Evidence Evaluation Benchmark), a diagnostic benchmark built on three principles: (1) focused visual perception of subtle targets in complex scenes, (2) traceable evidence via bounding box evaluation, and (3) second-order reasoning to test object interactions and spatial hierarchies beyond simple object localization. Prioritizing images with dense objects, we initially sample 1K high-quality images from SA-1B, and incorporate eight LMM experts to manually annotate questions, candidate options, and answers for each image. After three stages of quality control, TreeBench consists of 405 challenging visual question-answering pairs, even the most advanced models struggle with this benchmark, where none of them reach 60% accuracy, e.g., OpenAI-o3 scores only 54.87. Furthermore, we introduce TreeVGR (Traceable Evidence Enhanced Visual Grounded Reasoning), a training paradigm to supervise localization and reasoning jointly with reinforcement learning, enabling accurate localizations and explainable reasoning pathways. Initialized from Qwen2.5-VL-7B, it improves V* Bench (+16.8), MME-RealWorld (+12.6), and TreeBench (+13.4), proving traceability is key to advancing vision-grounded reasoning. The code is available at https://github.com/Haochen-Wang409/TreeVGR.

Abstract of Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology (arXiv:2507.07999, CC0)
Made by Wang et al. (TreeVGR authors) Released Jul 2025 Detection and captioning All tasks public Last in a launch post Dec 2025 , dropped before it saturated Last model added Jun 2026
Level
University Undergraduates and entry-level professionals
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
7
Models scored in lab cards
2
Launch posts list it, from 1 lab
7
System cards report it, from 3 labs

Seed2.1 Pro leads with 71.1% among models released by Jun 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in Jun 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 Seed2.1 Pro 71.1%
  2. 2 Seed2.0 Pro 64.7%
  3. 3 Nemotron 3 Nano Omni 30B-A3B 51.6%
  4. 4 GLM-4.6V 51.4%
  5. 5 GLM-4.5V 50.1%
  6. 6 Nemotron Nano 12B v2 VL 42.5%
  7. 7 GLM-4.1V-9B-Thinking 37.5%
#ModelScoreSettingDateSource
1 Seed2.1 Pro 71.1% Jun 2026 Model card
2 Seed2.0 Pro 64.7% Feb 2026 Model card
3 Nemotron 3 Nano Omni 30B-A3B 51.6% Apr 2026 Model card
4 GLM-4.6V 51.4% Dec 2025 Model card
5 GLM-4.5V 50.1% Aug 2025 Model card
6 Nemotron Nano 12B v2 VL 42.5% Nov 2025 Model card
7 GLM-4.1V-9B-Thinking 37.5% Jul 2025 Model card

5 models have held number one since its release in Jul 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Z.ai ByteDance Seed
Lab claims Not tested since Jun 2026
Seed2.1 Pro 71.1%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Seed2.1 Pro Seed2.0 Pro GLM-4.6V GLM-4.5V GLM-4.1V-9B-Thinking 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Seed2.1 Pro Seed2.0 Pro

Labs headline it less often now, last in Dec 2025

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Z.ai
DateLaunch postScoreAs the post writes it
8 Dec 2025 GLM-4.6V 51.4 TreeBench
11 Aug 2025 GLM-4.5V 50.1 TreeBench

7 system cards report it, from 3 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
23 Jun 2026 Seed2.1 Pro model card 71.1 TreeBench (Table 9) Capabilities
27 Apr 2026 Nemotron 3 Nano Omni 30B-A3B technical report 51.6 TreeBench, reasoning on, 43.7 reasoning off (Table 7) Capabilities
14 Feb 2026 Seed2.0 Pro model card 64.7 TreeBench (Table 8) Capabilities
1 Jan 2026 GLM-4.6V technical report 51.4 TreeBench, GLM-4.6V thinking (Table 2) Capabilities
6 Nov 2025 Nemotron Nano 12B v2 VL technical report 42.5 TreeBench, reasoning on (Table 3). Reasoning off 38.5 Capabilities
13 Aug 2025 GLM-4.5V technical report 50.1 TreeBench, GLM-4.5V thinking (Table 2) Capabilities
1 Jul 2025 GLM-4.1V-9B-Thinking technical report 37.5 TreeBench, GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%