TreeBench
Models answer 405 visual questions about small targets in dense SA-1B images and give bounding boxes as evidence. Scored by answer accuracy, with box overlap as a second measure.
At release no model reached 60% accuracy, and OpenAI o3 scored 54.87.
In the authors’ words
Models like OpenAI-o3 pioneer visual grounded reasoning by dynamically referencing visual regions, just like human "thinking with images". However, no benchmark exists to evaluate these capabilities holistically. To bridge this gap, we propose TreeBench (Traceable Evidence Evaluation Benchmark), a diagnostic benchmark built on three principles: (1) focused visual perception of subtle targets in complex scenes, (2) traceable evidence via bounding box evaluation, and (3) second-order reasoning to test object interactions and spatial hierarchies beyond simple object localization. Prioritizing images with dense objects, we initially sample 1K high-quality images from SA-1B, and incorporate eight LMM experts to manually annotate questions, candidate options, and answers for each image. After three stages of quality control, TreeBench consists of 405 challenging visual question-answering pairs, even the most advanced models struggle with this benchmark, where none of them reach 60% accuracy, e.g., OpenAI-o3 scores only 54.87. Furthermore, we introduce TreeVGR (Traceable Evidence Enhanced Visual Grounded Reasoning), a training paradigm to supervise localization and reasoning jointly with reinforcement learning, enabling accurate localizations and explainable reasoning pathways. Initialized from Qwen2.5-VL-7B, it improves V* Bench (+16.8), MME-RealWorld (+12.6), and TreeBench (+13.4), proving traceability is key to advancing vision-grounded reasoning. The code is available at https://github.com/Haochen-Wang409/TreeVGR.
- Level
- University Undergraduates and entry-level professionals
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Multiple choice
Seed2.1 Pro leads with 71.1% among models released by Jun 2026
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
The newest model with a score here came out in Jun 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 16 more
- 1
Seed2.1 Pro 71.1%
- 2
Seed2.0 Pro 64.7%
- 3
Nemotron 3 Nano Omni 30B-A3B 51.6%
- 4 GLM-4.6V 51.4%
- 5 GLM-4.5V 50.1%
- 6
Nemotron Nano 12B v2 VL 42.5%
- 7 GLM-4.1V-9B-Thinking 37.5%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | | 71.1% | Jun 2026 | Model card | |
| 2 | | 64.7% | Feb 2026 | Model card | |
| 3 | | 51.6% | Apr 2026 | Model card | |
| 4 | GLM-4.6V | 51.4% | Dec 2025 | Model card | |
| 5 | GLM-4.5V | 50.1% | Aug 2025 | Model card | |
| 6 | | 42.5% | Nov 2025 | Model card | |
| 7 | GLM-4.1V-9B-Thinking | 37.5% | Jul 2025 | Model card |
5 models have held number one since its release in Jul 2025
Lab claims count from the day the card came out, independent runs from the model’s release.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Labs headline it less often now, last in Dec 2025
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
7 system cards report it, from 3 labs
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 23 Jun 2026 | | 71.1 | TreeBench (Table 9) | Capabilities |
| 27 Apr 2026 | | 51.6 | TreeBench, reasoning on, 43.7 reasoning off (Table 7) | Capabilities |
| 14 Feb 2026 | | 64.7 | TreeBench (Table 8) | Capabilities |
| 1 Jan 2026 | GLM-4.6V technical report | 51.4 | TreeBench, GLM-4.6V thinking (Table 2) | Capabilities |
| 6 Nov 2025 | | 42.5 | TreeBench, reasoning on (Table 3). Reasoning off 38.5 | Capabilities |
| 13 Aug 2025 | GLM-4.5V technical report | 50.1 | TreeBench, GLM-4.5V thinking (Table 2) | Capabilities |
| 1 Jul 2025 | GLM-4.1V-9B-Thinking technical report | 37.5 | TreeBench, GLM-4.1V-9B-Thinking thinking (Table 2) | Capabilities |
More in Vision
- MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
- MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
- CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
- MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy.
Qwen 3.8 Max 97.7%
- ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance.
Command A Vision 90.9%
- DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers.
Seed1.5-VL 96.9%