Benchmark Almanac

VISTA

Gemini 2.5 Pro Mar 2025 54.6% leads since Mar 2025 Not in model cards

Models answer 758 single-turn prompts about one image each, covering perception and reasoning. Three LLM judges grade answers with yes/no rubric questions. The main score is average task accuracy.

Scale keeps the data private until models improve a lot. Judges were Claude 3.5 Sonnet, GPT-4o and Gemini 1.5 Pro, with three runs per evaluation. The board ranked 63 models on 2026-11-01.

Made by Scale AI Released Dec 2024 General visual questions Tasks kept private Last model added Mar 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
Leaderboard
54
Models scored on its own leaderboard

Gemini 2.5 Pro leads with 54.6% among models released by Mar 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in Mar 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

54 models. Scroll sideways for the rest.

  1. 54.6 Gemini 2.5 Pro
  2. 54.6 Gemini 2.5 Pro
  3. 53.9 GPT-5.4 Pro
  4. 52.4 GPT-5 Pro
  5. 51.8 o4-mini
  6. 51.6 o3-pro
  7. 51.5 Gemini 3 Pro
  8. 50.9 GPT-5.4
  9. 50.8 Gemini 2.5 Pro
  10. 50.4 GPT-5 mini
  11. 50.1 o3
  12. 49.7 GPT-5
  13. 49.1 Gemini 2.5 Flash
  14. 48.8 Claude Sonnet 4.5
  15. 48.4 Claude Opus 4.1
  16. 48.2 Claude 3.7 Sonnet
  17. 47.3 o1-pro
  18. 47.0 Gemini 2.5 Flash
  19. 47.0 Claude Opus 4
  20. 46.9 Gemini 3.1 Flash-Lite
  21. 46.6 GPT-5.2
  22. 46.4 Claude Opus 4.5
  23. 46.1 Claude Opus 4.6
  24. 45.5 Gemini 2.0 Flash Thinking
  25. 45.5 Claude Sonnet 4
  26. 45.3 GPT-4.1
  27. 45.3 o1
  28. 43.8 GPT-5.1
  29. 43.3 Gemini 2.0 Pro
  30. 42.1 GPT-4.5
  31. 41.9 Kimi K2.5
  32. 41.1 GPT-4.1 mini
  33. 40.0 Gemini 2.0 Flash
  34. 39.9 Gemini 2.0 Flash
  35. 38.7 Claude 3.5 Sonnet (October 2024)
  36. 38.4 Claude 3.5 Sonnet
  37. 38.3 Llama 4 Maverick
  38. 38.0 GPT-4o
  39. 37.1 Gemini 1.5 Pro
  40. 34.9 GPT-4o
  41. 34.9 GPT-5.1 Instant
  42. 34.6 Mistral Medium 3
  43. 34.0 Gemini 1.5 Flash
  44. 33.9 Pixtral Large
  45. 32.7 Gemini 2.0 Flash-Lite
  46. 28.6 Qwen2-VL-72B-Instruct
  47. 27.8 Claude 3 Opus
  48. 26.6 GPT-4.1 nano
  49. 26.3 Amazon Nova Pro
  50. 26.0 Pixtral 12B
  51. 25.5 Amazon Nova Lite
  52. 24.6 Llama 3.2 90B
  53. 20.5 Llama 3.2 11B
  54. 15.2 Phi-3.5-vision-instruct
  1. 1 Gemini 2.5 Pro Mar 2025 54.6%
  2. 2 Gemini 2.5 Pro Jun 2025 54.6%
  3. 3 GPT-5.4 Pro 53.9%
  4. 4 GPT-5 Pro 52.4%
  5. 5 o4-mini high 51.8%
  6. 6 o3-pro high 51.6%
  7. 7 Gemini 3 Pro 51.5%
  8. 8 GPT-5.4 xhigh 50.9%
  9. 9 Gemini 2.5 Pro May 2025 50.8%
  10. 10 GPT-5 mini 50.4%
  11. 11 o3 high 50.1%
  12. 12 GPT-5 49.7%
  13. 13 Gemini 2.5 Flash May 2025 49.1%
  14. 14 Claude Sonnet 4.5 thinking 48.8%
  15. 15 Claude Opus 4.1 thinking 48.4%
  16. 16 Claude 3.7 Sonnet thinking 48.2%
  17. 17 o1-pro 47.3%
  18. 18 Gemini 2.5 Flash Apr 2025 47%
  19. 19 Claude Opus 4 thinking 47%
  20. 20 Gemini 3.1 Flash-Lite 46.9%
  21. 21 GPT-5.2 46.6%
  22. 22 Claude Opus 4.5 thinking 46.4%
  23. 23 Claude Opus 4.6 max 46.1%
  24. 24 Gemini 2.0 Flash Thinking Jan 2025 45.5%
  25. 25 Claude Sonnet 4 thinking 45.5%
  26. 26 GPT-4.1 45.3%
  27. 27 o1 45.3%
  28. 28 GPT-5.1 thinking 43.8%
  29. 29 Gemini 2.0 Pro 43.3%
  30. 30 GPT-4.5 42.1%
  31. 31 Kimi K2.5 41.9%
  32. 32 GPT-4.1 mini 41.1%
  33. 33 Gemini 2.0 Flash Dec 2024 40%
  34. 34 Gemini 2.0 Flash Feb 2025 39.9%
  35. 35 Claude 3.5 Sonnet (October 2024) 38.7%
  36. 36 Claude 3.5 Sonnet 38.4%
  37. 37 Llama 4 Maverick 38.3%
  38. 38 GPT-4o Nov 2024 38%
  39. 39 Gemini 1.5 Pro Sep 2024 37.1%
  40. 40 GPT-4o Aug 2024 34.9%
  41. 41 GPT-5.1 Instant 34.9%
  42. 42 Mistral Medium 3 34.6%
  43. 43 Gemini 1.5 Flash Sep 2024 34%
  44. 44 Pixtral Large 33.9%
  45. 45 Gemini 2.0 Flash-Lite 32.7%
  46. 46 Qwen2-VL-72B-Instruct 28.6%
  47. 47 Claude 3 Opus 27.8%
  48. 48 GPT-4.1 nano 26.6%
  49. 49 Amazon Nova Pro 26.3%
  50. 50 Pixtral 12B 26%
  51. 51 Amazon Nova Lite 25.5%
  52. 52 Llama 3.2 90B 24.6%
  53. 53 Llama 3.2 11B 20.5%
  54. 54 Phi-3.5-vision-instruct 15.2%
#ModelScoreSettingDateSource
1 Gemini 2.5 Pro Mar 2025 54.6% Mar 2025 Official leaderboard
2 Gemini 2.5 Pro Jun 2025 54.6% Jun 2025 Official leaderboard
3 GPT-5.4 Pro 53.9% Mar 2026 Official leaderboard
4 GPT-5 Pro 52.4% Oct 2025 Official leaderboard
5 o4-mini 51.8% high Apr 2025 Official leaderboard
6 o3-pro 51.6% high Jun 2025 Official leaderboard
7 Gemini 3 Pro 51.5% Nov 2025 Official leaderboard
8 GPT-5.4 50.9% xhigh Mar 2026 Official leaderboard
9 Gemini 2.5 Pro May 2025 50.8% May 2025 Official leaderboard
10 GPT-5 mini 50.4% Aug 2025 Official leaderboard
11 o3 50.1% high Apr 2025 Official leaderboard
12 GPT-5 49.7% Aug 2025 Official leaderboard
13 Gemini 2.5 Flash May 2025 49.1% May 2025 Official leaderboard
14 Claude Sonnet 4.5 48.8% thinking Sep 2025 Official leaderboard
15 Claude Opus 4.1 48.4% thinking Aug 2025 Official leaderboard
16 Claude 3.7 Sonnet 48.2% thinking Feb 2025 Official leaderboard
17 o1-pro 47.3% Mar 2025 Official leaderboard
18 Gemini 2.5 Flash Apr 2025 47% Apr 2025 Official leaderboard
19 Claude Opus 4 47% thinking May 2025 Official leaderboard
20 Gemini 3.1 Flash-Lite 46.9% Mar 2026 Official leaderboard
21 GPT-5.2 46.6% Dec 2025 Official leaderboard
22 Claude Opus 4.5 46.4% thinking Nov 2025 Official leaderboard
23 Claude Opus 4.6 46.1% max Feb 2026 Official leaderboard
24 Gemini 2.0 Flash Thinking Jan 2025 45.5% Jan 2025 Official leaderboard
25 Claude Sonnet 4 45.5% thinking May 2025 Official leaderboard
26 GPT-4.1 45.3% Apr 2025 Official leaderboard
27 o1 45.3% Dec 2024 Official leaderboard
28 GPT-5.1 43.8% thinking Nov 2025 Official leaderboard
29 Gemini 2.0 Pro 43.3% Feb 2025 Official leaderboard
30 GPT-4.5 42.1% Feb 2025 Official leaderboard
31 Kimi K2.5 41.9% Jan 2026 Official leaderboard
32 GPT-4.1 mini 41.1% Apr 2025 Official leaderboard
33 Gemini 2.0 Flash Dec 2024 40% Dec 2024 Official leaderboard
34 Gemini 2.0 Flash Feb 2025 39.9% Feb 2025 Official leaderboard
35 Claude 3.5 Sonnet (October 2024) 38.7% Oct 2024 Official leaderboard
36 Claude 3.5 Sonnet 38.4% Jun 2024 Official leaderboard
37 Llama 4 Maverick 38.3% Apr 2025 Official leaderboard
38 GPT-4o Nov 2024 38% Nov 2024 Official leaderboard
39 Gemini 1.5 Pro Sep 2024 37.1% Sep 2024 Official leaderboard
40 GPT-4o Aug 2024 34.9% Aug 2024 Official leaderboard
41 GPT-5.1 Instant 34.9% Nov 2025 Official leaderboard
42 Mistral Medium 3 34.6% May 2025 Official leaderboard
43 Gemini 1.5 Flash Sep 2024 34% Aug 2024 Official leaderboard
44 Pixtral Large 33.9% Nov 2024 Official leaderboard
45 Gemini 2.0 Flash-Lite 32.7% Feb 2025 Official leaderboard
46 Qwen2-VL-72B-Instruct 28.6% Aug 2024 Official leaderboard
47 Claude 3 Opus 27.8% Feb 2024 Official leaderboard
48 GPT-4.1 nano 26.6% Apr 2025 Official leaderboard
49 Amazon Nova Pro 26.3% Dec 2024 Official leaderboard
50 Pixtral 12B 26% Sep 2024 Official leaderboard
51 Amazon Nova Lite 25.5% Dec 2024 Official leaderboard
52 Llama 3.2 90B 24.6% Sep 2024 Official leaderboard
53 Llama 3.2 11B 20.5% Sep 2024 Official leaderboard
54 Phi-3.5-vision-instruct 15.2% Aug 2024 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
o4-mini
medium
51.8%
o3
medium
50.1%
Claude Sonnet 4.5
thinking
48.8%
Claude Opus 4.1
thinking
48.4%
Claude 3.7 Sonnet
thinking
48.2%
Claude Opus 4
thinking
47%
Claude Opus 4.5
thinking
46.4%
Claude Opus 4.6
non-thinking
46.1%
Claude Sonnet 4
thinking
45.5%
ModelEffortHarnessScoreCost per taskOutput tokens
Gemini 2.5 Pro Mar 2025 54.6%
Gemini 2.5 Pro Jun 2025 54.6%
GPT-5.4 Pro 53.9%
GPT-5 Pro 52.4%
o4-mini high 51.8%
o4-mini medium 51.7%
o3-pro high 51.6%
Gemini 3 Pro 51.5%
GPT-5.4 xhigh 50.9%
Gemini 2.5 Pro May 2025 50.8%
GPT-5 mini 50.4%
o3 high 50.1%
GPT-5 49.7%
o3 medium 49.6%
Gemini 2.5 Flash May 2025 49.1%
Claude Sonnet 4.5 thinking 48.8%
Claude Opus 4.1 thinking 48.4%
Claude 3.7 Sonnet thinking 48.2%
o1-pro 47.3%
Gemini 2.5 Flash Apr 2025 47%
Claude Opus 4 thinking 47%
Gemini 3.1 Flash-Lite 46.9%
GPT-5.2 46.6%
Claude Opus 4.5 thinking 46.4%
Claude Opus 4.6 max 46.1%
Gemini 2.0 Flash Thinking Jan 2025 45.5%
Claude Sonnet 4 thinking 45.5%
Claude Opus 4.6 non-thinking 45.5%
GPT-4.1 45.3%
Claude Opus 4.5 45.3%
o1 45.3%
Claude Opus 4.1 45.3%
Claude Sonnet 4.5 45%
GPT-5.1 thinking 43.8%
Claude Opus 4 43.5%
Gemini 2.0 Pro 43.3%
Claude Sonnet 4 43.2%
Claude 3.7 Sonnet 43%
GPT-4.5 42.1%
Kimi K2.5 41.9%
GPT-4.1 mini 41.1%
Gemini 2.0 Flash Dec 2024 40%
Gemini 2.0 Flash Feb 2025 39.9%
Claude 3.5 Sonnet (October 2024) 38.7%
Claude 3.5 Sonnet 38.4%
Llama 4 Maverick 38.3%
GPT-4o Nov 2024 38%
Gemini 1.5 Pro Sep 2024 37.1%
GPT-4o Aug 2024 34.9%
GPT-5.1 Instant 34.9%
Mistral Medium 3 34.6%
Gemini 1.5 Flash Sep 2024 34%
Pixtral Large 33.9%
Gemini 2.0 Flash-Lite 32.7%
Qwen2-VL-72B-Instruct 28.6%
Claude 3 Opus 27.8%
GPT-4.1 nano 26.6%
Amazon Nova Pro 26.3%
Pixtral 12B 26%
Amazon Nova Lite 25.5%
Llama 3.2 90B 24.6%
Llama 3.2 11B 20.5%
Phi-3.5-vision-instruct 15.2%

6 models have held number one since its release in Dec 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic Google DeepMind OpenAI
Own leaderboard Not tested since Mar 2026
Gemini 2.5 Pro 54.6%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemini 2.5 Pro Claude 3.7 Sonnet Gemini 2.0 Flash Claude 3.5 Sonnet Claude 3 Opus 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Gemini 2.5 Pro Claude 3.7 Sonnet

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%