Benchmark Almanac

LogicVista

Qwen 3.8 Max 91.9% leads since Aug 2026 In use

Models answer 448 multiple-choice questions on spatial, deductive, inductive, numeric, and mechanical reasoning set in images. Scored by accuracy.

In the authors’ words

We propose LogicVista, an evaluation benchmark that assesses the integrated logical reasoning capabilities of multimodal large language models (MLLMs) in Visual contexts. Recent advancements in MLLMs have demonstrated various fascinating abilities, from crafting poetry based on an image to performing mathematical reasoning. However, there is still a lack of systematic evaluation of MLLMs' proficiency in logical reasoning tasks, which are essential for activities like navigation and puzzle-solving. Thus we evaluate general logical cognition abilities across 5 logical reasoning tasks encompassing 9 different capabilities, using a sample of 448 multiple-choice questions. Each question is annotated with the correct answer and the human-written reasoning behind the selection, enabling both open-ended and multiple-choice evaluation. A total of 8 MLLMs are comprehensively evaluated using LogicVista. Code and Data Available at https://github.com/Yijia-Xiao/LogicVista.

Abstract of LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts (arXiv:2407.04973, CC0)
Made by UCLA Released Jul 2024 Visual math and diagrams All tasks public Last in a launch post Aug 2026 Last model added Aug 2026
Level
School Students, from grade school to high school
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Multiple choice
Yijia-Xiao/LogicVista 18 stars Paper logicvista.github.io
12
Models scored in lab cards
7
Launch posts list it, from 6 labs
10
System cards report it, from 7 labs

Qwen 3.8 Max leads with 91.9% among models released by Aug 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in Aug 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 91.9 Qwen 3.8 Max
  2. 81.9 Seed2.0 Pro
  3. 78.3 Seed1.8
  4. 73.8 EXAONE 4.5 33B
  5. 72.2 Qwen3-VL-235B-A22B
  6. 66.9 STEP3-VL-10B
  7. 66.7 InternVL3.5-241B-A28B
  8. 62.4 GLM-4.5V
  9. 62.1 GLM-4.6V
  10. 60.4 GLM-4.1V-9B-Thinking
  11. 58.4 Nemotron Nano 12B v2 VL
  12. 37.4 LFM2.5-VL-3B
  1. 1 Qwen 3.8 Max 91.9%
  2. 2 Seed2.0 Pro 81.9%
  3. 3 Seed1.8 78.3%
  4. 4 EXAONE 4.5 33B 73.8%
  5. 5 Qwen3-VL-235B-A22B 72.2%
  6. 6 STEP3-VL-10B 66.9%
  7. 7 InternVL3.5-241B-A28B 66.7%
  8. 8 GLM-4.5V 62.4%
  9. 9 GLM-4.6V 62.1%
  10. 10 GLM-4.1V-9B-Thinking 60.4%
  11. 11 Nemotron Nano 12B v2 VL 58.4%
  12. 12 LFM2.5-VL-3B 37.4%
#ModelScoreSettingDateSource
1 Qwen 3.8 Max 91.9% Aug 2026 Model card
2 Seed2.0 Pro 81.9% Feb 2026 Model card
3 Seed1.8 78.3% Dec 2025 Model card
4 EXAONE 4.5 33B 73.8% Apr 2026 Model card
5 Qwen3-VL-235B-A22B 72.2% Nov 2025 Model card
6 STEP3-VL-10B 66.9% Jan 2026 Model card
7 InternVL3.5-241B-A28B 66.7% Aug 2025 Model card
8 GLM-4.5V 62.4% Aug 2025 Model card
9 GLM-4.6V 62.1% Jan 2026 Model card
10 GLM-4.1V-9B-Thinking 60.4% Jul 2025 Model card
11 Nemotron Nano 12B v2 VL 58.4% Nov 2025 Model card
12 LFM2.5-VL-3B 37.4% Aug 2026 Model card

7 models have held number one since its release in Jul 2024

Lab claims count from the day the card came out, independent runs from the model’s release.

Z.ai Shanghai AI Lab Qwen ByteDance Seed
Lab claims Passed 85% in Aug 2026, by Qwen 3.8 Max
Qwen 3.8 Max 91.9%

The best score passed 85% in Aug 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Qwen 3.8 Max Seed2.0 Pro Qwen3-VL-235B-A22B InternVL3.5-241B-A28B GLM-4.5V 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Qwen 3.8 Max Seed2.0 Pro

2 labs put it in a launch post in the last six months

Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.

Z.ai
Shanghai AI Lab
ByteDance Seed
LG AI Research
Qwen
Liquid AI
DateLaunch postScoreAs the post writes it
12 Aug 2026 LFM2.5-VL-3B 37.4 LogicVista
3 Aug 2026 Qwen 3.8 Max 91.9 LogicVista
9 Apr 2026 EXAONE 4.5 33B 73.8 LogicVista
14 Feb 2026 Seed2.0 Pro 81.9 LogicVista
18 Dec 2025 Seed1.8 78.3 LogicVista
25 Aug 2025 InternVL3.5-241B-A28B 66.7 LogicVista
11 Aug 2025 GLM-4.5V 62.4 LogicVista

10 system cards report it, from 7 labs

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
9 Apr 2026 EXAONE 4.5 33B technical report 73.8 LogicVista (Table 2, Reasoning mode) Capabilities
21 Mar 2026 Seed1.8 model card 78.3 LogicVista (Table 2) Capabilities
14 Feb 2026 Seed2.0 Pro model card 81.9 LogicVista (Table 8) Capabilities
14 Jan 2026 STEP3-VL-10B technical report 66.9 LogicVista, SeRe (Tables 1 and 3). PaCoRe 71.36 Capabilities
1 Jan 2026 GLM-4.6V technical report 62.1 LogicVista, GLM-4.6V thinking (Table 2) Capabilities
26 Nov 2025 Qwen3-VL-235B-A22B technical report 72.2 LogicVista (Table 2) Capabilities
6 Nov 2025 Nemotron Nano 12B v2 VL technical report 58.4 LogicVista, reasoning on (Table 3). Reasoning off 38.7 Capabilities
25 Aug 2025 InternVL3.5-241B-A28B technical report 66.7 LogicVista (Table 2, Table 3). With parallel thinking 67.6 Capabilities
13 Aug 2025 GLM-4.5V technical report 62.4 LogicVista, GLM-4.5V thinking (Table 2) Capabilities
1 Jul 2025 GLM-4.1V-9B-Thinking technical report 60.4 LogicVista, GLM-4.1V-9B-Thinking thinking (Table 2) Capabilities

More in Vision

All 234 in Vision
  • MMMU In use 11.5K college-level multimodal questions across 30 subjects. Multiple-choice and open accuracy. GPT-5.1 85.4%
  • MMMU-Pro In use Harder MMMU: college-level multimodal questions with more options and vision-only inputs to remove text shortcuts. Kimi K3.1 Pro 84.1%
  • CharXiv In use Chart understanding on real arXiv figures. Reasoning questions require synthesizing information from complex charts. Claude Fable 5 93.5%
  • MATH-Vision (MathVision) In use 3,040 competition math problems with visual context. Answer accuracy. Qwen 3.8 Max 97.7%
  • ChartQA In use Models answer human-written and generated questions about chart images that need visual and arithmetic reasoning. Scored by relaxed accuracy with a 5 percent numeric tolerance. Command A Vision 90.9%
  • DocVQA In use Models answer 50,000 questions about 12,000 scanned document images by reading their text and layout. Scoring is ANLS, a string similarity to reference answers. Seed1.5-VL 96.9%