Benchmark Almanac

RealWorldQA

Qwen3.8 Flash 88.5% leads since Aug 2026 In use

Models answer a question about each of 700+ real-world photos, many taken from vehicles, that test basic spatial understanding. Answers are easy to verify.

Made by xAI Released Apr 2024 Spatial reasoning and counting All tasks public Last in a launch post Sep 2026 Last model added Sep 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
Dataset x.ai
30
Models scored in lab cards
23
Launch posts list it, from 7 labs
17
System cards report it, from 9 labs

Qwen3.8 Flash leads with 88.5%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

30 models. Scroll sideways for the rest.

  1. 88.5 Qwen3.8 Flash
  2. 88.0 Qwen 3.8 Max
  3. 87.7 Qwen3.8-Omni-Flash
  4. 86.9 Qwen3.7-Plus
  5. 86.7 Seed2.1 Pro
  6. 86.0 Seed2.0 Pro
  7. 85.9 Qwen 3.8 27B
  8. 85.4 Qwen 3.6 Plus
  9. 85.3 Qwen 3.6 35B-A3B
  10. 85.1 Qwen3.5-122B-A10B
  11. 84.1 Qwen3.5-Omni-Plus
  12. 84.1 Qwen3.6 27B
  13. 83.9 Qwen3.5 397B-A17B
  14. 81.3 Qwen3-VL-235B-A22B
  15. 78.4 Seed1.5-VL
  16. 77.8 Seed1.6-Thinking
  17. 75.7 Qwen2.5-VL-72B
  18. 75.2 InternVL3.5-241B-A28B
  19. 74.8 LongCat-Flash-Omni
  20. 74.4 STEP3-VL-10B
  21. 73.1 LFM2.5-VL-3B
  22. 72.2 Nemotron Nano 12B v2 VL
  23. 70.4 Gemini 1.5 Pro
  24. 70.3 Qwen2.5-Omni-7B
  25. 70.0 Kimi-VL-A3B-Thinking-2506
  26. 65.8 LFM2-VL-1.6B
  27. 62.2 North Micro Vision Instruct
  28. 53.9 Gemma 3 27B
  29. 52.2 Gemma 3 12B
  30. 45.5 Gemma 3 4B
  1. 1 Qwen3.8 Flash 88.5%
  2. 2 Qwen 3.8 Max 88%
  3. 3 Qwen3.8-Omni-Flash 87.7%
  4. 4 Qwen3.7-Plus 86.9%
  5. 5 Seed2.1 Pro 86.7%
  6. 6 Seed2.0 Pro 86%
  7. 7 Qwen 3.8 27B 85.9%
  8. 8 Qwen 3.6 Plus 85.4%
  9. 9 Qwen 3.6 35B-A3B 85.3%
  10. 10 Qwen3.5-122B-A10B 85.1%
  11. 11 Qwen3.5-Omni-Plus 84.1%
  12. 12 Qwen3.6 27B 84.1%
  13. 13 Qwen3.5 397B-A17B 83.9%
  14. 14 Qwen3-VL-235B-A22B 81.3%
  15. 15 Seed1.5-VL 78.4%
  16. 16 Seed1.6-Thinking 77.8%
  17. 17 Qwen2.5-VL-72B 75.7%
  18. 18 InternVL3.5-241B-A28B 75.2%
  19. 19 LongCat-Flash-Omni 74.8%
  20. 20 STEP3-VL-10B 74.4%
  21. 21 LFM2.5-VL-3B 73.1%
  22. 22 Nemotron Nano 12B v2 VL 72.2%
  23. 23 Gemini 1.5 Pro May 2024 70.4%
  24. 24 Qwen2.5-Omni-7B 70.3%
  25. 25 Kimi-VL-A3B-Thinking-2506 70%
  26. 26 LFM2-VL-1.6B 65.8%
  27. 27 North Micro Vision Instruct 62.2%
  28. 28 Gemma 3 27B 53.9%
  29. 29 Gemma 3 12B 52.2%
  30. 30 Gemma 3 4B 45.5%
#ModelScoreSettingDateSource
1 Qwen3.8 Flash 88.5% Aug 2026 Model card
2 Qwen 3.8 Max 88% Aug 2026 Model card
3 Qwen3.8-Omni-Flash 87.7% Sep 2026 Model card
4 Qwen3.7-Plus 86.9% Jun 2026 Model card
5 Seed2.1 Pro 86.7% Jun 2026 Model card
6 Seed2.0 Pro 86% Feb 2026 Model card
7 Qwen 3.8 27B 85.9% Aug 2026 Model card
8 Qwen 3.6 Plus 85.4% Apr 2026 Model card
9 Qwen 3.6 35B-A3B 85.3% Apr 2026 Model card
10 Qwen3.5-122B-A10B 85.1% Feb 2026 Model card
11 Qwen3.5-Omni-Plus 84.1% Mar 2026 Model card
12 Qwen3.6 27B 84.1% Apr 2026 Model card
13 Qwen3.5 397B-A17B 83.9% Feb 2026 Model card
14 Qwen3-VL-235B-A22B 81.3% Sep 2025 Model card
15 Seed1.5-VL 78.4% May 2025 Model card
16 Seed1.6-Thinking 77.8% Jun 2025 Model card
17 Qwen2.5-VL-72B 75.7% Feb 2025 Model card
18 InternVL3.5-241B-A28B 75.2% Aug 2025 Model card
19 LongCat-Flash-Omni 74.8% Oct 2025 Model card
20 STEP3-VL-10B 74.4% Jan 2026 Model card
21 LFM2.5-VL-3B 73.1% Aug 2026 Model card
22 Nemotron Nano 12B v2 VL 72.2% Nov 2025 Model card
23 Gemini 1.5 Pro May 2024 70.4% May 2024 Model card
24 Qwen2.5-Omni-7B 70.3% Mar 2025 Model card
25 Kimi-VL-A3B-Thinking-2506 70% Jun 2025 Model card
26 LFM2-VL-1.6B 65.8% Nov 2025 Model card
27 North Micro Vision Instruct 62.2% Aug 2026 Model card
28 Gemma 3 27B 53.9% Mar 2025 Model card
29 Gemma 3 12B 52.2% Mar 2025 Model card
30 Gemma 3 4B 45.5% Mar 2025 Model card

8 models have held number one since its release in Apr 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Qwen ByteDance Seed
Lab claims Passed 85% in Feb 2026, by Seed2.0 Pro
Qwen3.8 Flash 88.5%

The best score passed 85% in Feb 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen3.8 Flash Seed2.0 Pro Qwen3-VL-235B-A22B Qwen2.5-VL-72B Gemini 1.5 Pro 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Qwen3.8 Flash Qwen3-VL-235B-A22B

3 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Google DeepMind
Qwen
ByteDance Seed
Moonshot AI
Cohere
Liquid AI
Meituan
DateLaunch postScoreAs the post writes it
18 Sep 2026 Qwen3.8-Omni-Flash 87.7 RealWorldQA
26 Aug 2026 Qwen3.8 Flash 88.5 Real-world perception RealWorldQA
14 Aug 2026 Qwen 3.8 27B 85.9 Real-world perception RealWorldQA
12 Aug 2026 LFM2.5-VL-3B 73.1 RealWorldQA
12 Aug 2026 North Micro Vision Instruct 62.2 RealWorldQA
3 Aug 2026 Qwen 3.8 Max 88 RealWorldQA
1 Jun 2026 Qwen3.7-Plus 86.9 RealWorldQA
22 Apr 2026 Qwen3.6 27B 84.1 RealWorldQA
15 Apr 2026 Qwen 3.6 35B-A3B 85.3 RealWorldQA
2 Apr 2026 Qwen 3.6 Plus 85.4 RealWorldQA
30 Mar 2026 Qwen3.5-Omni-Plus 84.1 RealWorldQA
24 Feb 2026 Qwen3.5-122B-A10B 85.1 RealWorldQA
16 Feb 2026 Qwen3.5 397B-A17B 83.9 RealWorldQA
31 Oct 2025 LongCat-Flash-Omni 74.8 RealWorldQA
23 Sep 2025 Qwen3-VL-235B-A22B 81.3 RealWorldQA
12 Aug 2025 LFM2-VL-1.6B 65.2 RealWorldQA
31 Jul 2025 Command A Vision RealWorldQA (chart only)
25 Jun 2025 Seed1.6-Thinking 77.8 RealWorldQA
21 Jun 2025 Kimi-VL-A3B-Thinking-2506 70 RealWorldQA (Acc)
12 May 2025 Seed1.5-VL 78.4 RealWorldQA
27 Mar 2025 Qwen2.5-Omni-7B 70.3 RealWorldQAavg
12 Mar 2025 Gemma 3 27B 53.9 RealWorldQA (PT)
14 May 2024 Gemini 1.5 Pro 70.4 Realworld QA (test), 0-shot

17 system cards report it, from 9 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
22 Sep 2026 Qwen3.8-Omni-Flash technical report 87.7 RealWorldQA (Table 3) Capabilities
23 Jun 2026 Seed2.1 Pro model card 86.7 RealWorldQA (Table 9) Capabilities
17 Apr 2026 Qwen3.5-Omni-Plus technical report 84.1 RealWorldQA (Table 6) Capabilities
14 Feb 2026 Seed2.0 Pro model card 86 RealWorldQA (Table 8) Capabilities
14 Jan 2026 STEP3-VL-10B technical report 74.4 RealWorldQA, SeRe (Tables 1 and 3). PaCoRe 75.56 Capabilities
28 Nov 2025 LFM2-VL-1.6B technical report 65.8 RealWorldQA, LFM2-VL-1.6B (Table 8) Capabilities
26 Nov 2025 Qwen3-VL-235B-A22B technical report 81.3 RealWorldQA (Table 2) Capabilities
6 Nov 2025 Nemotron Nano 12B v2 VL technical report 72.2 RealWorldQA, reasoning on (Table 3). Reasoning off 76.2 Capabilities
31 Oct 2025 LongCat-Flash-Omni technical report 74.8 RealWorldQA (Table 5) Capabilities
25 Aug 2025 InternVL3.5-241B-A28B technical report 75.2 RealWorldQA (Table 5) Capabilities
23 Jun 2025 Kimi-VL-A3B-Thinking-2506 technical report 70 RealWorldQA (Acc) (Table 5) Capabilities
11 May 2025 Seed1.5-VL technical report 78.4 RealWorldQA, thinking (Table 6). Non-thinking 77.0 Capabilities
26 Mar 2025 Qwen2.5-Omni-7B technical report 70.3 RealWorldQA avg (Table 5) Capabilities
25 Mar 2025 Gemma 3 4B technical report 45.5 RealWorldQA, PT, 4-shot (Table 11) Capabilities
25 Mar 2025 Gemma 3 27B technical report 53.9 RealWorldQA, PT, 4-shot (Table 11) Capabilities
25 Mar 2025 Gemma 3 12B technical report 52.2 RealWorldQA, PT, 4-shot (Table 11) Capabilities
19 Feb 2025 Qwen2.5-VL-72B technical report 75.7 RealWorldQA avg (Table 3) Capabilities

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%