Benchmark Almanac

ParseBench

LlamaParse Agentic Plus 86.6% tops its leaderboard Claude Opus 5.5 78.9% leads the models we track Not in model cards

Document parsers process about 2,000 human-verified enterprise pages. Over 169,000 test rules score tables, charts, content faithfulness, formatting and visual grounding.

On Hugging Face's official benchmark list, with 168 likes.

In the authors’ words

AI agents are changing the requirements for document parsing. What matters is semantic correctness: parsed output must preserve the structure and meaning needed for autonomous decisions, including correct table structure, precise chart data, semantically meaningful formatting, and visual grounding. Existing benchmarks do not fully capture this setting for enterprise automation, relying on narrow document distributions and text-similarity metrics that miss agent-critical failures. We introduce ParseBench, a benchmark of ~2,000 human-verified pages from enterprise documents spanning insurance, finance, and government, organized around five capability dimensions: tables, charts, content faithfulness, semantic formatting, and visual grounding. Across 14 methods spanning vision-language models, specialized document parsers, and LlamaParse, the benchmark reveals a fragmented capability landscape: no method is consistently strong across all five dimensions. LlamaParse Agentic achieves the highest overall score at 84.9%, and the benchmark highlights the remaining capability gaps across current systems. Dataset and evaluation code are available on https://huggingface.co/datasets/llamaindex/ParseBench and https://github.com/run-llama/ParseBench.

Abstract of ParseBench: A Document Parsing Benchmark for AI Agents (arXiv:2604.08538, CC0)
Made by LlamaIndex Released Apr 2026 Documents, charts and text in images All tasks public Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
run-llama/ParseBench 620 stars Paper Dataset Leaderboard parsebench.ai
111
Entries on its own leaderboard, 42 models we track

LlamaParse Agentic Plus tops the leaderboard with 86.6%

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders. The best model we track is Claude Opus 5.5, at 78.9%.

111 entries. Scroll sideways for the rest.

  1. 86.6 LlamaParse Agentic Plus
  2. 83.1 LlamaParse Agentic
  3. 81.6 Pulse Ultra 2
  4. 80.5 Upstage Document Parse 2
  5. 80.3 anyformat
  6. 78.9 Claude Opus 5.5
  7. 78.3 oi-parser
  8. 78.1 Claude Fable 5.1
  9. 78.0 LlamaParse Cost Effective
  10. 77.2 rakedoc-nano
  11. 76.8 aeddix-alpine-ocr
  12. 76.7 florin-parser-nano
  13. 76.4 Nutrient DWS
  14. 75.3 Extend
  15. 75.2 Nemotron-3.5-Parse
  16. 75.1 LightOnOCR-3-4B
  17. 75.0 Gemini 3 Flash
  18. 74.7 Databricks AI Parse
  19. 74.6 LightOnOCR-3-0.8B
  20. 74.5 GPT-5.6 Sol
  21. 74.3 Infinity-Parser2-Pro
  22. 73.6 GPT-6.1 Terra
  23. 73.4 Gemini 3.5 Flash
  24. 73.3 Extend Light
  25. 73.0 Datalab Accurate
  26. 73.0 Reducto
  27. 72.9 Datalab Balanced
  28. 72.8 MinerU2.5-Pro-2605-1.2B
  29. 72.4 KDL-Frontier-Parser-nano
  30. 72.2 Infinity-Parser2-Flash
  31. 71.8 OvisOCR3
  32. 71.5 Gemini 3.8 Flash
  33. 71.4 LightOnOCR-3-1B
  34. 70.8 GLM-5.3-Flash
  35. 70.7 Gemini 3.7 Flash
  36. 70.1 Chandra-ocr-2
  37. 69.9 GPT-6 Astra
  38. 69.8 Claude Fable 5
  39. 69.7 Qwen 3.8 27B
  40. 69.7 GPT-6.1 Sol
  41. 69.4 Gemini 3.6 Flash
  42. 69.3 Claude Sonnet 5.5
  43. 69.1 Gemini 3.1 Pro
  44. 68.9 GPT-6 Sol
  45. 68.2 Mistral OCR 4
  46. 67.8 Datalab Fast
  47. 67.4 PaddleOCR-VL-1.6
  48. 67.1 GPT-5.6 Terra
  49. 66.9 GPT-5.6 Luna
  50. 66.6 GPT-5.5
  51. 66.0 PaddleOCR-VL-1.5
  52. 64.8 Surya OCR 2
  53. 64.6 V4.1 Flash
  54. 64.3 GPT-6 Luna
  55. 63.3 Claude Opus 4.7
  56. 62.6 HunyuanOCR 1.5
  57. 62.4 Gemma 4 31B IT
  58. 62.3 Claude Opus 4.8
  59. 62.2 GPT-5.4
  60. 62.0 Qwen3-VL-8B-Instruct
  61. 61.7 MiniCPM-OCR 1.0
  62. 61.2 Claude Sonnet 5
  63. 60.7 DocAI
  64. 59.7 Gemini 3.5 Flash-Lite
  65. 59.6 Azure Document Intelligence
  66. 58.5 Gemma-4-26B-A4B-it
  67. 58.3 Gemini 3.1 Flash-Lite
  68. 57.5 TeleOCR
  69. 57.1 Unlimited-OCR-2
  70. 56.1 Claude Haiku 5.5
  71. 55.8 Dots.mocr
  72. 54.1 Claude Opus 4.6
  73. 53.5 PyMuPDF4LLM
  74. 53.4 HPD-Parsing
  75. 53.1 Claude Haiku 4.5
  76. 52.1 Falcon-OCR
  77. 51.5 GPT-5 mini
  78. 50.6 Docling-models
  79. 50.4 Google Cloud Document AI
  80. 48.5 Nemotron 3 Nano Omni 30B-A3B
  81. 48.0 LightOnOCR-2-1B
  82. 47.9 AWS Textract
  83. 46.2 Qianfan-OCR
  84. 46.2 Unlimited-OCR
  85. 45.9 Jina-OCR-v1
  86. 45.9 MinerU2.5-2509-1.2B
  87. 45.2 LandingAI
  88. 44.1 Qwen 3.6 35B-A3B
  89. 43.4 GPT-5.4 Nano
  90. 42.6 MinerU-Diffusion-V1-0320-2.5B
  91. 41.2 DeepSeek-OCR-2
  92. 40.9 PaddleOCR-VL
  93. 40.8 OvisOCR2
  94. 40.5 Gemma 4 E4B
  95. 40.4 Nova 2 Lite
  96. 40.2 Warp Ingest
  97. 39.5 Granite Vision 4.1 4B
  98. 38.9 WeVisDoc-2B
  99. 36.9 LiteParse
  100. 36.1 WeVisDoc-4B
  101. 35.4 Qwen3.5-4B
  102. 31.9 Qwen3.5-9B
  103. 31.1 Firecrawl
  104. 29.6 GLM-OCR
  105. 29.4 OpenDataLoader
  106. 28.4 Qwen3.5-0.8B
  107. 27.3 Qwen3.5-2B
  108. 26.6 pdf-inspector
  109. 18.6 MarkItDown
  110. 16.6 PyMuPDF
  111. 14.9 pypdf
  1. 1 LlamaParse Agentic Plus 86.6%
  2. 2 LlamaParse Agentic 83.1%
  3. 3 Pulse Ultra 2 81.6%
  4. 4 Upstage Document Parse 2 enhanced 80.5%
  5. 5 anyformat standard 80.3%
  6. 6 Claude Opus 5.5 high 78.9%
  7. 7 oi-parser 78.3%
  8. 8 Claude Fable 5.1 78.1%
  9. 9 LlamaParse Cost Effective 78%
  10. 10 rakedoc-nano 77.2%
  11. 11 aeddix-alpine-ocr kdl pipeline 76.8%
  12. 12 florin-parser-nano 76.7%
  13. 13 Nutrient DWS agentic 76.4%
  14. 14 Extend 2.0 75.3%
  15. 15 Nemotron-3.5-Parse 75.2%
  16. 16 LightOnOCR-3-4B 75.1%
  17. 17 Gemini 3 Flash high 75%
  18. 18 Databricks AI Parse 74.7%
  19. 19 LightOnOCR-3-0.8B 74.6%
  20. 20 GPT-5.6 Sol high 74.5%
  21. 21 Infinity-Parser2-Pro 74.3%
  22. 22 GPT-6.1 Terra medium 73.6%
  23. 23 Gemini 3.5 Flash high 73.4%
  24. 24 Extend Light 1.0 73.3%
  25. 25 Datalab Accurate 73%
  26. 26 Reducto agentic 73%
  27. 27 Datalab Balanced 72.9%
  28. 28 MinerU2.5-Pro-2605-1.2B 72.8%
  29. 29 KDL-Frontier-Parser-nano 72.4%
  30. 30 Infinity-Parser2-Flash 72.2%
  31. 31 OvisOCR3 71.8%
  32. 32 Gemini 3.8 Flash high 71.5%
  33. 33 LightOnOCR-3-1B 71.4%
  34. 34 GLM-5.3-Flash 70.8%
  35. 35 Gemini 3.7 Flash high 70.7%
  36. 36 Chandra-ocr-2 70.1%
  37. 37 GPT-6 Astra low 69.9%
  38. 38 Claude Fable 5 69.8%
  39. 39 Qwen 3.8 27B thinking 69.7%
  40. 40 GPT-6.1 Sol low 69.7%
  41. 41 Gemini 3.6 Flash high 69.4%
  42. 42 Claude Sonnet 5.5 69.3%
  43. 43 Gemini 3.1 Pro 69.1%
  44. 44 GPT-6 Sol high 68.9%
  45. 45 Mistral OCR 4 annotation 68.2%
  46. 46 Datalab Fast 67.8%
  47. 47 PaddleOCR-VL-1.6 67.4%
  48. 48 GPT-5.6 Terra medium 67.1%
  49. 49 GPT-5.6 Luna xhigh 66.9%
  50. 50 GPT-5.5 medium 66.6%
  51. 51 PaddleOCR-VL-1.5 66%
  52. 52 Surya OCR 2 64.8%
  53. 53 V4.1 Flash high 64.6%
  54. 54 GPT-6 Luna max 64.3%
  55. 55 Claude Opus 4.7 63.3%
  56. 56 HunyuanOCR 1.5 62.6%
  57. 57 Gemma 4 31B IT 62.4%
  58. 58 Claude Opus 4.8 62.3%
  59. 59 GPT-5.4 none 62.2%
  60. 60 Qwen3-VL-8B-Instruct 62%
  61. 61 MiniCPM-OCR 1.0 61.7%
  62. 62 Claude Sonnet 5 61.2%
  63. 63 DocAI 60.7%
  64. 64 Gemini 3.5 Flash-Lite high 59.7%
  65. 65 Azure Document Intelligence layout 59.6%
  66. 66 Gemma-4-26B-A4B-it 58.5%
  67. 67 Gemini 3.1 Flash-Lite 58.3%
  68. 68 TeleOCR 57.5%
  69. 69 Unlimited-OCR-2 57.1%
  70. 70 Claude Haiku 5.5 56.1%
  71. 71 Dots.mocr 55.8%
  72. 72 Claude Opus 4.6 54.1%
  73. 73 PyMuPDF4LLM 53.5%
  74. 74 HPD-Parsing 53.4%
  75. 75 Claude Haiku 4.5 thinking 53.1%
  76. 76 Falcon-OCR 52.1%
  77. 77 GPT-5 mini medium 51.5%
  78. 78 Docling-models 50.6%
  79. 79 Google Cloud Document AI 50.4%
  80. 80 Nemotron 3 Nano Omni 30B-A3B thinking 48.5%
  81. 81 LightOnOCR-2-1B 48%
  82. 82 AWS Textract 47.9%
  83. 83 Qianfan-OCR 46.2%
  84. 84 Unlimited-OCR 46.2%
  85. 85 Jina-OCR-v1 45.9%
  86. 86 MinerU2.5-2509-1.2B 45.9%
  87. 87 LandingAI 45.2%
  88. 88 Qwen 3.6 35B-A3B 44.1%
  89. 89 GPT-5.4 Nano 43.4%
  90. 90 MinerU-Diffusion-V1-0320-2.5B 42.6%
  91. 91 DeepSeek-OCR-2 41.2%
  92. 92 PaddleOCR-VL 40.9%
  93. 93 OvisOCR2 40.8%
  94. 94 Gemma 4 E4B 40.5%
  95. 95 Nova 2 Lite 40.4%
  96. 96 Warp Ingest 40.2%
  97. 97 Granite Vision 4.1 4B 39.5%
  98. 98 WeVisDoc-2B 38.9%
  99. 99 LiteParse no ocr 36.9%
  100. 100 WeVisDoc-4B 36.1%
  101. 101 Qwen3.5-4B 35.4%
  102. 102 Qwen3.5-9B 31.9%
  103. 103 Firecrawl 31.1%
  104. 104 GLM-OCR 29.6%
  105. 105 OpenDataLoader 29.4%
  106. 106 Qwen3.5-0.8B 28.4%
  107. 107 Qwen3.5-2B 27.3%
  108. 108 pdf-inspector 26.6%
  109. 109 MarkItDown 18.6%
  110. 110 PyMuPDF html 16.6%
  111. 111 pypdf 14.9%
#EntryScoreSettingDateSource
1 LlamaParse Agentic Plus 86.6% Official leaderboard
2 LlamaParse Agentic 83.1% Official leaderboard
3 Pulse Ultra 2 81.6% Official leaderboard
4 Upstage Document Parse 2 80.5% enhanced Official leaderboard
5 anyformat 80.3% standard Official leaderboard
6 Claude Opus 5.5 78.9% high Sep 2026 Official leaderboard
7 oi-parser 78.3% Official leaderboard
8 Claude Fable 5.1 78.1% Sep 2026 Official leaderboard
9 LlamaParse Cost Effective 78% Official leaderboard
10 rakedoc-nano 77.2% Official leaderboard
11 aeddix-alpine-ocr 76.8% kdl pipeline Official leaderboard
12 florin-parser-nano 76.7% Official leaderboard
13 Nutrient DWS 76.4% agentic Official leaderboard
14 Extend 75.3% 2.0 Official leaderboard
15 Nemotron-3.5-Parse 75.2% Official leaderboard
16 LightOnOCR-3-4B 75.1% Official leaderboard
17 Gemini 3 Flash 75% high Dec 2025 Official leaderboard
18 Databricks AI Parse 74.7% Official leaderboard
19 LightOnOCR-3-0.8B 74.6% Official leaderboard
20 GPT-5.6 Sol 74.5% high Jul 2026 Official leaderboard
21 Infinity-Parser2-Pro 74.3% Official leaderboard
22 GPT-6.1 Terra 73.6% medium Oct 2026 Official leaderboard
23 Gemini 3.5 Flash 73.4% high May 2026 Official leaderboard
24 Extend Light 73.3% 1.0 Official leaderboard
25 Datalab Accurate 73% Official leaderboard
26 Reducto 73% agentic Official leaderboard
27 Datalab Balanced 72.9% Official leaderboard
28 MinerU2.5-Pro-2605-1.2B 72.8% Official leaderboard
29 KDL-Frontier-Parser-nano 72.4% Official leaderboard
30 Infinity-Parser2-Flash 72.2% Official leaderboard
31 OvisOCR3 71.8% Official leaderboard
32 Gemini 3.8 Flash 71.5% high Sep 2026 Official leaderboard
33 LightOnOCR-3-1B 71.4% Official leaderboard
34 GLM-5.3-Flash 70.8% Aug 2026 Official leaderboard
35 Gemini 3.7 Flash 70.7% high Aug 2026 Official leaderboard
36 Chandra-ocr-2 70.1% Official leaderboard
37 GPT-6 Astra 69.9% low Sep 2026 Official leaderboard
38 Claude Fable 5 69.8% Jun 2026 Official leaderboard
39 Qwen 3.8 27B 69.7% thinking Aug 2026 Official leaderboard
40 GPT-6.1 Sol 69.7% low Sep 2026 Official leaderboard
41 Gemini 3.6 Flash 69.4% high Jul 2026 Official leaderboard
42 Claude Sonnet 5.5 69.3% Sep 2026 Official leaderboard
43 Gemini 3.1 Pro 69.1% Feb 2026 Official leaderboard
44 GPT-6 Sol 68.9% high Sep 2026 Official leaderboard
45 Mistral OCR 4 68.2% annotation Official leaderboard
46 Datalab Fast 67.8% Official leaderboard
47 PaddleOCR-VL-1.6 67.4% Official leaderboard
48 GPT-5.6 Terra 67.1% medium Jul 2026 Official leaderboard
49 GPT-5.6 Luna 66.9% xhigh Jul 2026 Official leaderboard
50 GPT-5.5 66.6% medium Apr 2026 Official leaderboard
51 PaddleOCR-VL-1.5 66% Official leaderboard
52 Surya OCR 2 64.8% Official leaderboard
53 V4.1 Flash 64.6% high Official leaderboard
54 GPT-6 Luna 64.3% max Sep 2026 Official leaderboard
55 Claude Opus 4.7 63.3% Apr 2026 Official leaderboard
56 HunyuanOCR 1.5 62.6% Official leaderboard
57 Gemma 4 31B IT 62.4% Apr 2026 Official leaderboard
58 Claude Opus 4.8 62.3% May 2026 Official leaderboard
59 GPT-5.4 62.2% none Mar 2026 Official leaderboard
60 Qwen3-VL-8B-Instruct 62% Official leaderboard
61 MiniCPM-OCR 1.0 61.7% Official leaderboard
62 Claude Sonnet 5 61.2% Jun 2026 Official leaderboard
63 DocAI 60.7% Official leaderboard
64 Gemini 3.5 Flash-Lite 59.7% high Jul 2026 Official leaderboard
65 Azure Document Intelligence 59.6% layout Official leaderboard
66 Gemma-4-26B-A4B-it 58.5% Official leaderboard
67 Gemini 3.1 Flash-Lite 58.3% Mar 2026 Official leaderboard
68 TeleOCR 57.5% Official leaderboard
69 Unlimited-OCR-2 57.1% Official leaderboard
70 Claude Haiku 5.5 56.1% Oct 2026 Official leaderboard
71 Dots.mocr 55.8% Official leaderboard
72 Claude Opus 4.6 54.1% Feb 2026 Official leaderboard
73 PyMuPDF4LLM 53.5% Official leaderboard
74 HPD-Parsing 53.4% Official leaderboard
75 Claude Haiku 4.5 53.1% thinking Oct 2025 Official leaderboard
76 Falcon-OCR 52.1% Official leaderboard
77 GPT-5 mini 51.5% medium Aug 2025 Official leaderboard
78 Docling-models 50.6% Official leaderboard
79 Google Cloud Document AI 50.4% Official leaderboard
80 Nemotron 3 Nano Omni 30B-A3B 48.5% thinking Apr 2026 Official leaderboard
81 LightOnOCR-2-1B 48% Official leaderboard
82 AWS Textract 47.9% Official leaderboard
83 Qianfan-OCR 46.2% Official leaderboard
84 Unlimited-OCR 46.2% Official leaderboard
85 Jina-OCR-v1 45.9% Official leaderboard
86 MinerU2.5-2509-1.2B 45.9% Official leaderboard
87 LandingAI 45.2% Official leaderboard
88 Qwen 3.6 35B-A3B 44.1% Apr 2026 Official leaderboard
89 GPT-5.4 Nano 43.4% Mar 2026 Official leaderboard
90 MinerU-Diffusion-V1-0320-2.5B 42.6% Official leaderboard
91 DeepSeek-OCR-2 41.2% Official leaderboard
92 PaddleOCR-VL 40.9% Official leaderboard
93 OvisOCR2 40.8% Official leaderboard
94 Gemma 4 E4B 40.5% Mar 2026 Official leaderboard
95 Nova 2 Lite 40.4% Dec 2025 Official leaderboard
96 Warp Ingest 40.2% Official leaderboard
97 Granite Vision 4.1 4B 39.5% Official leaderboard
98 WeVisDoc-2B 38.9% Official leaderboard
99 LiteParse 36.9% no ocr Official leaderboard
100 WeVisDoc-4B 36.1% Official leaderboard
101 Qwen3.5-4B 35.4% Feb 2026 Official leaderboard
102 Qwen3.5-9B 31.9% Feb 2026 Official leaderboard
103 Firecrawl 31.1% Official leaderboard
104 GLM-OCR 29.6% Jan 2026 Official leaderboard
105 OpenDataLoader 29.4% Official leaderboard
106 Qwen3.5-0.8B 28.4% Feb 2026 Official leaderboard
107 Qwen3.5-2B 27.3% Feb 2026 Official leaderboard
108 pdf-inspector 26.6% Official leaderboard
109 MarkItDown 18.6% Official leaderboard
110 PyMuPDF 16.6% html Official leaderboard
111 pypdf 14.9% Official leaderboard

The best score, 78.9%, costs $6.13 a task

Runs on the cost frontier score higher than any run that's cheaper. Gemini 3 Flash at medium comes within 4.7 points of the best for $1.98 a task, 3 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Google DeepMind Z.ai Anthropic OpenAI Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 $100 Cost per task, log scale Claude Opus 5.5, high Gemini 3 Flash, medium GLM-5.3-Flash GPT-6 Luna, xhigh GPT-6 Luna, low GPT-6 Luna, none 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 $100 Cost per task, log scale Claude Opus 5.5, high Gemini 3 Flash, minimal
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 high 78.9% $6.13
Claude Fable 5.1 78.1% $16
Claude Opus 5.5 77% $5.79
Gemini 3 Flash high 75% $2.41
GPT-5.6 Sol high 74.5% $8.08
Gemini 3 Flash medium 74.2% $1.98
GPT-6.1 Terra medium 73.6% $1.09
Claude Opus 5.5 low 73.6% $5.46
Gemini 3.5 Flash high 73.4% $4.95
Gemini 3.8 Flash high 71.5% $3.41
Gemini 3 Flash minimal 71% $0.65
GLM-5.3-Flash 70.8% $0.44
Gemini 3.7 Flash high 70.7% $1.91
Gemini 3.7 Flash medium 70.1% $1.06
Gemini 3.8 Flash medium 69.9% $1.24
GPT-6 Astra low 69.9% $9.87
Claude Fable 5 69.8% $16
GPT-6.1 Terra low 69.8% $0.96
Qwen 3.8 27B thinking 69.7%
GPT-6.1 Sol low 69.7% $1.95
Gemini 3.6 Flash high 69.4% $4.25
Gemini 3.8 Flash low 69.3% $0.71
Claude Sonnet 5.5 69.3% $2.74
Gemini 3.5 Flash medium 69.2% $4.28
Gemini 3.1 Pro 69.1% $8.49
GPT-6 Sol high 68.9% $3.47
Gemini 3.7 Flash low 68.7% $0.70
GPT-6 Sol medium 67.4% $2.40
GPT-5.6 Terra medium 67.1% $2.71
GPT-5.6 Luna xhigh 66.9% $1.01
GPT-5.6 Luna max 66.8% $1.70
GPT-5.6 Terra low 66.7% $2.59
GPT-5.5 medium 66.6% $13
Gemini 3.6 Flash medium 66.1% $3.76
GPT-5.6 Luna high 66% $0.59
GPT-6 Sol none 65.3% $1.86
Gemini 3.6 Flash minimal 64.3% $1.48
GPT-6 Luna max 64.3% $0.72
Claude Opus 4.7 63.3% $7.14
GPT-6 Luna xhigh 63.1% $0.29
GPT-6 Luna high 63% $0.21
GPT-5.6 Terra none 62.8% $2.36
GPT-5.5 none 62.8% $5.93
Gemma 4 31B IT 62.4%
Claude Opus 4.8 62.3% $7.30
GPT-5.4 none 62.2% $2.90
Gemini 3.5 Flash minimal 62% $1.82
GPT-5.6 Luna medium 61.9% $0.29
Claude Sonnet 5 61.2% $3.18
GPT-6 Luna medium 61.1% $0.14
GPT-5.6 Sol none 60.9% $4.06
Qwen 3.8 27B 60.3%
Gemini 3.1 Pro low 59.9% $2.50
Gemini 3.5 Flash-Lite high 59.7% $1.18
GPT-5.6 Luna low 58.5% $0.23
Gemini 3.1 Flash-Lite 58.3% $0.29
GPT-6 Luna low 58% $0.11
Gemini 3.1 Flash-Lite medium 56.8% $0.32
Gemini 3.1 Flash-Lite high 56.3% $1.04
Claude Haiku 5.5 56.1% $0.14
Gemini 3.5 Flash-Lite 55.5% $0.56
GPT-5.6 Luna none 54.5% $0.22
Claude Opus 4.6 54.1% $5.78
Claude Haiku 4.5 thinking 53.1% $3.69
Gemini 3.5 Flash-Lite medium 52.2% $0.47
GPT-5 mini medium 51.5% $3.05
GPT-6 Luna none 51.4% $0.09
Nemotron 3 Nano Omni 30B-A3B thinking 48.5%
GPT-5 mini minimal 46.8% $0.88
Claude Haiku 4.5 disable thinking 45.2% $1.64
Qwen 3.6 35B-A3B 44.1%
GPT-5.4 Nano 43.4% $0.25
Gemma 4 E4B 40.5%
Nova 2 Lite 40.4% $0.70
Qwen3.5-4B 35.4%
Qwen3.5-9B 31.9%
GLM-OCR 29.6%
Qwen3.5-0.8B 28.4%
Qwen3.5-2B 27.3%

3 models have held number one since its release in Apr 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Anthropic
Own leaderboard 3 leaders
Claude Opus 5.5 78.9%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Opus 5.5 Gemini 3 Flash Claude Haiku 4.5 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Claude Opus 5.5 Claude Haiku 4.5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%