Benchmark Almanac

DocVQA

Seed1.5-VL 96.9% leads since May 2025 In use

Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers.

Test answers stay on the evaluation server, and top models now pass the 94.36 human score in the paper.

In the authors’ words

We present a new dataset for Visual Question Answering (VQA) on document images called DocVQA. The dataset consists of 50,000 questions defined on 12,000+ document images. Detailed analysis of the dataset in comparison with similar datasets for VQA and reading comprehension is presented. We report several baseline results by adopting existing VQA and reading comprehension models. Although the existing models perform reasonably well on certain types of questions, there is large performance gap compared to human performance (94.36% accuracy). The models need to improve specifically on questions where understanding structure of the document is crucial. The dataset, code and leaderboard are available at docvqa.org

Abstract of DocVQA: A Dataset for VQA on Document Images (arXiv:2007.00398, CC0)
Made by CVC , Universitat Autònoma de Barcelona and IIIT Hyderabad Released Jul 2020 Documents, charts and text in images All tasks public Last in a launch post Sep 2026 Last model added Sep 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
Paper docvqa.org
33
Models scored in lab cards
30
Launch posts list it, from 15 labs
20
System cards report it, from 10 labs

Seed1.5-VL leads with 96.9% among models released by Sep 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in Sep 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

33 models. Scroll sideways for the rest.

  1. 96.9 Seed1.5-VL
  2. 96.5 Qwen3-VL-235B-A22B
  3. 96.4 Qwen2.5-VL-72B
  4. 95.9 Command A Vision
  5. 95.6 Nemotron 3 Nano Omni 30B-A3B
  6. 95.6 DeepSeek V4.1 Flash
  7. 95.5 ERNIE 5.0
  8. 95.3 Mistral Medium 3
  9. 95.2 Claude 3.5 Sonnet
  10. 95.2 Qwen2.5-Omni-7B
  11. 94.9 InternVL3.5-241B-A28B
  12. 94.9 Mistral Small 3.2
  13. 94.4 Llama 4 Maverick
  14. 94.4 Nemotron Nano 12B v2 VL
  15. 94.2 Claude 3.5 Sonnet (October 2024)
  16. 94.1 Mistral Small 3.1
  17. 93.6 Grok-2
  18. 93.6 ERNIE-4.5-VL-28B-A3B-Thinking
  19. 93.2 Phi-4-multimodal
  20. 93.1 Gemini 1.5 Pro
  21. 92.8 GPT-4o
  22. 92.1 North Micro Vision Instruct
  23. 91.8 LongCat-Flash-Omni
  24. 91.2 Llama-3.1-Nemotron-Nano-VL-8B-V1
  25. 91.1 LFM2.5-VL-3B
  26. 90.9 Gemini 1.0 Ultra
  27. 89.5 Claude 3 Sonnet
  28. 89.3 Claude 3 Opus
  29. 88.8 Claude 3 Haiku
  30. 87.1 Gemma 3 12B
  31. 86.6 Gemma 3 27B
  32. 86.5 Gemini 1.5 Pro
  33. 75.8 Gemma 3 4B
  1. 1 Seed1.5-VL 96.9%
  2. 2 Qwen3-VL-235B-A22B 96.5%
  3. 3 Qwen2.5-VL-72B 96.4%
  4. 4 Command A Vision 95.9%
  5. 5 Nemotron 3 Nano Omni 30B-A3B 95.6%
  6. 6 DeepSeek V4.1 Flash 95.6%
  7. 7 ERNIE 5.0 95.5%
  8. 8 Mistral Medium 3 95.3%
  9. 9 Claude 3.5 Sonnet 95.2%
  10. 10 Qwen2.5-Omni-7B 95.2%
  11. 11 InternVL3.5-241B-A28B 94.9%
  12. 12 Mistral Small 3.2 94.9%
  13. 13 Llama 4 Maverick 94.4%
  14. 14 Nemotron Nano 12B v2 VL 94.4%
  15. 15 Claude 3.5 Sonnet (October 2024) 94.2%
  16. 16 Mistral Small 3.1 94.1%
  17. 17 Grok-2 Aug 2024 93.6%
  18. 18 ERNIE-4.5-VL-28B-A3B-Thinking 93.6%
  19. 19 Phi-4-multimodal 93.2%
  20. 20 Gemini 1.5 Pro May 2024 93.1%
  21. 21 GPT-4o May 2024 92.8%
  22. 22 North Micro Vision Instruct 92.1%
  23. 23 LongCat-Flash-Omni 91.8%
  24. 24 Llama-3.1-Nemotron-Nano-VL-8B-V1 91.2%
  25. 25 LFM2.5-VL-3B 91.1%
  26. 26 Gemini 1.0 Ultra 90.9%
  27. 27 Claude 3 Sonnet 89.5%
  28. 28 Claude 3 Opus 89.3%
  29. 29 Claude 3 Haiku 88.8%
  30. 30 Gemma 3 12B 87.1%
  31. 31 Gemma 3 27B 86.6%
  32. 32 Gemini 1.5 Pro Feb 2024 86.5%
  33. 33 Gemma 3 4B 75.8%
#ModelScoreSettingDateSource
1 Seed1.5-VL 96.9% May 2025 Model card
2 Qwen3-VL-235B-A22B 96.5% Sep 2025 Model card
3 Qwen2.5-VL-72B 96.4% Jan 2025 Model card
4 Command A Vision 95.9% Jul 2025 Model card
5 Nemotron 3 Nano Omni 30B-A3B 95.6% Apr 2026 Model card
6 DeepSeek V4.1 Flash 95.6% Sep 2026 Model card
7 ERNIE 5.0 95.5% Feb 2026 Model card
8 Mistral Medium 3 95.3% May 2025 Model card
9 Claude 3.5 Sonnet 95.2% Jun 2024 Model card
10 Qwen2.5-Omni-7B 95.2% Mar 2025 Model card
11 InternVL3.5-241B-A28B 94.9% Aug 2025 Model card
12 Mistral Small 3.2 94.9% Jun 2025 Model card
13 Llama 4 Maverick 94.4% Apr 2025 Model card
14 Nemotron Nano 12B v2 VL 94.4% Oct 2025 Model card
15 Claude 3.5 Sonnet (October 2024) 94.2% Oct 2024 Model card
16 Mistral Small 3.1 94.1% Mar 2025 Model card
17 Grok-2 Aug 2024 93.6% Aug 2024 Model card
18 ERNIE-4.5-VL-28B-A3B-Thinking 93.6% Nov 2025 Model card
19 Phi-4-multimodal 93.2% Feb 2025 Model card
20 Gemini 1.5 Pro May 2024 93.1% May 2024 Model card
21 GPT-4o May 2024 92.8% May 2024 Model card
22 North Micro Vision Instruct 92.1% Aug 2026 Model card
23 LongCat-Flash-Omni 91.8% Oct 2025 Model card
24 Llama-3.1-Nemotron-Nano-VL-8B-V1 91.2% Jun 2025 Model card
25 LFM2.5-VL-3B 91.1% Aug 2026 Model card
26 Gemini 1.0 Ultra 90.9% Dec 2023 Model card
27 Claude 3 Sonnet 89.5% Mar 2024 Model card
28 Claude 3 Opus 89.3% Mar 2024 Model card
29 Claude 3 Haiku 88.8% Mar 2024 Model card
30 Gemma 3 12B 87.1% Mar 2025 Model card
31 Gemma 3 27B 86.6% Mar 2025 Model card
32 Gemini 1.5 Pro Feb 2024 86.5% Feb 2024 Model card
33 Gemma 3 4B 75.8% Mar 2025 Model card

6 models have held number one since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind OpenAI Anthropic Qwen ByteDance Seed
Lab claims Passed 85% in Dec 2023, by Gemini 1.0 Ultra
Seed1.5-VL 96.9%

The best score passed 85% in Dec 2023

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Seed1.5-VL Claude 3.5 Sonnet Gemini 1.0 Ultra 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Seed1.5-VL

3 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

OpenAI
Google DeepMind
Anthropic
xAI
Qwen
Microsoft
Baidu
Mistral AI
Meta
ByteDance Seed
NVIDIA
Cohere
Meituan
Liquid AI
DeepSeek
DateLaunch postScoreAs the post writes it
10 Sep 2026 DeepSeek V4.1 Flash 95.6 DocVQA (LLM-Judge)
12 Aug 2026 LFM2.5-VL-3B 91.1 DocVQA (val)
12 Aug 2026 North Micro Vision Instruct 92.1 DocVQA VAL
6 Feb 2026 ERNIE 5.0 95.5 DocVQA (val)
11 Nov 2025 ERNIE-4.5-VL-28B-A3B-Thinking 93.6 DocVQA (val)
31 Oct 2025 LongCat-Flash-Omni 91.8 DocVQA
28 Oct 2025 Nemotron Nano 12B v2 VL 94.4 DocVQA (val)
23 Sep 2025 Qwen3-VL-235B-A22B 96.5 DocVQA_TEST
31 Jul 2025 Command A Vision 95.9 DocVQA
20 Jun 2025 Mistral Small 3.2 94.9 DocVQA
3 Jun 2025 Llama-3.1-Nemotron-Nano-VL-8B-V1 91.2 DocVQA val
12 May 2025 Seed1.5-VL 96.9 DocVQA
7 May 2025 Mistral Medium 3 95.3 DocVQA 0-shot
5 Apr 2025 Llama 4 Scout 94.4 DocVQA (test), anls
5 Apr 2025 Llama 4 Maverick 94.4 DocVQA (test), anls
27 Mar 2025 Qwen2.5-Omni-7B 95.2 DocVQAtest
17 Mar 2025 Mistral Small 3.1 94.1 DocVQA
16 Mar 2025 ERNIE 4.5 DocVQA
12 Mar 2025 Gemma 3 27B 86.6 DocVQA (IT); PT DocVQA (val) = 85.6
26 Feb 2025 Phi-4-multimodal 93.2 DocVQA
28 Jan 2025 Qwen2.5-VL-72B 96.4 DocVQA_VAL
22 Oct 2024 Claude 3.5 Sonnet (October 2024) 94.2 DocVQA (test, ANLS score)
13 Aug 2024 Grok-2 93.6 DocVQA
20 Jun 2024 Claude 3.5 Sonnet 95.2 DocVQA (test, ANLS score)
14 May 2024 Gemini 1.5 Pro 93.1 DocVQA (test), 0-shot (ANLS, with Cloud OCR annotations)
13 May 2024 GPT-4o 92.8 DocVQA (%) (test)
4 Mar 2024 Claude 3 Opus 89.3 DocVQA (test, ANLS score)
15 Feb 2024 Gemini 1.5 Pro 86.5 DocVQA (test), 0-shot (ANLS)
6 Dec 2023 Gemini 1.0 Ultra 90.9 DocVQA (test)
14 Mar 2023 GPT-4 88.4 DocVQA ANLS score (test) 0-shot (pixel-only)

20 system cards report it, from 10 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
27 Apr 2026 Nemotron 3 Nano Omni 30B-A3B technical report 95.6 DocVQA (Test), reasoning on, 93.3 reasoning off (Table 7) Capabilities
4 Feb 2026 ERNIE 5.0 technical report 95.5 DocVQAval (Table 3) Capabilities
26 Nov 2025 Qwen3-VL-235B-A22B technical report 96.5 DocVQA test (Table 2) Capabilities
6 Nov 2025 Nemotron Nano 12B v2 VL technical report 93.2 DocVQA (Test), reasoning on (Table 3). Reasoning off 94.7 Capabilities
31 Oct 2025 LongCat-Flash-Omni technical report 91.8 DocVQA (Table 5) Capabilities
25 Aug 2025 InternVL3.5-241B-A28B technical report 94.9 DocVQA (test) (Table 4) Capabilities
11 May 2025 Seed1.5-VL technical report 96.9 DocVQA, thinking (Table 6). Non-thinking 96.7 Capabilities
5 Apr 2025 Llama 4 Scout model card 94.4 DocVQA (test), 0-shot, anls, instruction tuned (Instruction tuned models table) Capabilities
5 Apr 2025 Llama 4 Maverick model card 94.4 DocVQA (test), 0-shot, anls, instruction tuned (Instruction tuned models table) Capabilities
26 Mar 2025 Qwen2.5-Omni-7B technical report 95.2 DocVQA test (Table 5) Capabilities
25 Mar 2025 Gemma 3 4B technical report 75.8 DocVQA, IT, P&S applied (Table 16) Capabilities
25 Mar 2025 Gemma 3 27B technical report 86.6 DocVQA, IT, P&S applied (Table 16) Capabilities
25 Mar 2025 Gemma 3 12B technical report 87.1 DocVQA, IT, P&S applied (Table 16) Capabilities
3 Mar 2025 Phi-4-multimodal technical report 93.2 DocVQA (test) (Table 1). Speech-prompt ShareGPT4o DocVQA 87.3 (Table 2) Capabilities
19 Feb 2025 Qwen2.5-VL-72B technical report 96.4 DocVQA test (Table 5) Capabilities
22 Oct 2024 Claude 3.5 Sonnet (October 2024) model card 94.2 DocVQA (test, ANLS score), 0-shot (Table 5) Capabilities
20 Jun 2024 Claude 3.5 Sonnet model card 95.2 DocVQA (test, ANLS score), 0-shot (Table 2) Capabilities
4 Mar 2024 Claude 3 Sonnet model card 89.5 DocVQA (test, ANLS score), 0-shot (Table 3) Capabilities
4 Mar 2024 Claude 3 Opus model card 89.3 DocVQA (test, ANLS score), 0-shot (Table 3) Capabilities
4 Mar 2024 Claude 3 Haiku model card 88.8 DocVQA (test, ANLS score), 0-shot (Table 3) Capabilities

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • MMStar In use Models answer 1,500 human-curated image questions that cannot be solved without the image, across 6 core capabilities. Scored by accuracy. Qwen 3.8 Max 85.9%