Benchmark Almanac

SAGE (Vals)

Claude Opus 4.7 56.1% leads since Apr 2026 Not in model cards

Models grade handwritten undergraduate work in PDEs, linear algebra and computer science against rubrics. Scores compare their deductions with expert graders.

v1 scored the share of submissions where every rubric check was right, which rewarded graders that never deduct. Board frozen 2026-10-08 with 56 models.

Made by Vals AI Released Oct 2025 Other Tasks kept private Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
University Undergraduates and entry-level professionals
Scored
Graded by people Human experts grade or compare the outputs.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
Leaderboard
81
Models scored on its own leaderboard

Claude Opus 4.7 leads with 56.1%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

81 models. Scroll sideways for the rest.

  1. 56.1 Claude Opus 4.7
  2. 55.0 Gemma 4 31B IT
  3. 54.8 Claude Opus 4.8
  4. 53.6 Gemini 4 Argon
  5. 52.8 Kimi K3
  6. 52.6 GPT-5.6 Sol
  7. 52.4 Qwen 3.8 27B
  8. 52.1 Claude Opus 4.5
  9. 51.9 Claude Fable 5
  10. 51.9 Gemini 3 Flash
  11. 51.8 Claude Sonnet 5.5
  12. 51.6 Claude Opus 4.6
  13. 51.5 GPT-5.5
  14. 51.3 Qwen 3.8 Max
  15. 50.8 GPT-5.4 Mini
  16. 50.6 MiniMax-M3
  17. 50.2 Kimi K2.6
  18. 49.9 Gemini 3.5 Flash
  19. 49.9 Kimi K2.5
  20. 49.5 Gemini 3.1 Flash-Lite
  21. 49.4 Claude Opus 5
  22. 49.3 GPT-5.2
  23. 49.2 Gemini 3.7 Flash
  24. 48.9 Claude Sonnet 5
  25. 48.7 Gemini 3.6 Flash
  26. 48.7 Gemini 3.1 Pro
  27. 48.5 Claude Fable 5.1
  28. 48.1 GPT-6 Luna
  29. 47.9 DeepSeek V4.1 Flash
  30. 47.7 Muse Spark 1.2
  31. 47.6 Gemini 3 Pro
  32. 47.3 Gemini 3.5 Flash-Lite
  33. 47.0 GPT-5.6 Terra
  34. 46.6 Claude Sonnet 4.6
  35. 46.5 GPT-6.1 Sol
  36. 46.4 GPT-6 Astra
  37. 45.8 Claude Opus 5.5
  38. 45.8 Claude Haiku 5.5
  39. 45.6 Qwen3.6 27B
  40. 45.6 Muse Spark 1.1
  41. 45.0 MiMo-V2.6 Pro
  42. 44.9 Qwen 3.6 Plus
  43. 44.8 GPT-6 Sol
  44. 44.8 Gemini 2.5 Flash
  45. 44.2 GPT-5.6 Luna
  46. 44.1 Gemini 2.5 Flash
  47. 43.7 GPT-5
  48. 43.5 MiMo-V2.6 Flash
  49. 43.3 GPT-5.4
  50. 43.3 MiMo-V2.5
  51. 43.2 GPT-5.1
  52. 43.0 GPT-5 mini
  53. 42.5 Qwen 3.5 Flash (hosted 35B-A3B)
  54. 42.1 Qwen3-VL-Plus
  55. 41.9 Gemini 2.5 Pro
  56. 41.8 o3
  57. 41.1 o4-mini
  58. 40.8 Grok 4.7
  59. 39.3 Qwen3.7-Plus
  60. 38.2 Grok 4.20
  61. 38.1 GPT-5.4 Nano
  62. 37.6 Mistral Medium 3.5
  63. 36.5 Inkling
  64. 36.1 Claude Sonnet 4.5
  65. 35.7 Llama 4 Maverick
  66. 35.1 Gemini 3.8 Flash
  67. 35.0 Claude Sonnet 4
  68. 35.0 Grok 4.5
  69. 34.8 Llama 4 Scout
  70. 32.7 Inkling-Small
  71. 31.8 Claude Haiku 4.5
  72. 31.4 Claude Opus 4.1
  73. 31.3 Grok 4.1 Fast
  74. 30.8 Gemini 2.5 Flash-Lite
  75. 30.4 Qwen 3.5 Plus (hosted 397B-A17B)
  76. 30.4 GPT-5 nano
  77. 29.8 Grok 4 Fast
  78. 28.9 Grok 4.6
  79. 25.1 Grok 4
  80. 24.6 Mistral Large 3
  81. 19.7 Grok 4.3
  1. 1 Claude Opus 4.7 56.1%
  2. 2 Gemma 4 31B IT 55%
  3. 3 Claude Opus 4.8 54.8%
  4. 4 Gemini 4 Argon 53.6%
  5. 5 Kimi K3 52.8%
  6. 6 GPT-5.6 Sol 52.6%
  7. 7 Qwen 3.8 27B 52.4%
  8. 8 Claude Opus 4.5 thinking 52.1%
  9. 9 Claude Fable 5 51.9%
  10. 10 Gemini 3 Flash 51.9%
  11. 11 Claude Sonnet 5.5 51.8%
  12. 12 Claude Opus 4.6 thinking 51.6%
  13. 13 GPT-5.5 51.5%
  14. 14 Qwen 3.8 Max 51.3%
  15. 15 GPT-5.4 Mini 50.8%
  16. 16 MiniMax-M3 50.6%
  17. 17 Kimi K2.6 50.2%
  18. 18 Gemini 3.5 Flash 49.9%
  19. 19 Kimi K2.5 thinking 49.9%
  20. 20 Gemini 3.1 Flash-Lite 49.5%
  21. 21 Claude Opus 5 49.4%
  22. 22 GPT-5.2 49.3%
  23. 23 Gemini 3.7 Flash 49.2%
  24. 24 Claude Sonnet 5 48.9%
  25. 25 Gemini 3.6 Flash 48.7%
  26. 26 Gemini 3.1 Pro 48.7%
  27. 27 Claude Fable 5.1 48.5%
  28. 28 GPT-6 Luna 48.1%
  29. 29 DeepSeek V4.1 Flash 47.9%
  30. 30 Muse Spark 1.2 47.7%
  31. 31 Gemini 3 Pro 47.6%
  32. 32 Gemini 3.5 Flash-Lite 47.3%
  33. 33 GPT-5.6 Terra 47%
  34. 34 Claude Sonnet 4.6 46.6%
  35. 35 GPT-6.1 Sol 46.5%
  36. 36 GPT-6 Astra 46.4%
  37. 37 Claude Opus 5.5 45.8%
  38. 38 Claude Haiku 5.5 45.8%
  39. 39 Qwen3.6 27B 45.6%
  40. 40 Muse Spark 1.1 45.6%
  41. 41 MiMo-V2.6 Pro 45%
  42. 42 Qwen 3.6 Plus 44.9%
  43. 43 GPT-6 Sol 44.8%
  44. 44 Gemini 2.5 Flash Jun 2025, non-thinking 44.8%
  45. 45 GPT-5.6 Luna 44.2%
  46. 46 Gemini 2.5 Flash Sep 2025, non-thinking 44.1%
  47. 47 GPT-5 43.7%
  48. 48 MiMo-V2.6 Flash 43.5%
  49. 49 GPT-5.4 xhigh 43.3%
  50. 50 MiMo-V2.5 43.3%
  51. 51 GPT-5.1 43.2%
  52. 52 GPT-5 mini 43%
  53. 53 Qwen 3.5 Flash (hosted 35B-A3B) 42.5%
  54. 54 Qwen3-VL-Plus 42.1%
  55. 55 Gemini 2.5 Pro Jun 2025 41.9%
  56. 56 o3 41.8%
  57. 57 o4-mini 41.1%
  58. 58 Grok 4.7 40.8%
  59. 59 Qwen3.7-Plus 39.3%
  60. 60 Grok 4.20 reasoning 38.2%
  61. 61 GPT-5.4 Nano 38.1%
  62. 62 Mistral Medium 3.5 37.6%
  63. 63 Inkling 36.5%
  64. 64 Claude Sonnet 4.5 thinking 36.1%
  65. 65 Llama 4 Maverick 35.7%
  66. 66 Gemini 3.8 Flash 35.1%
  67. 67 Claude Sonnet 4 non-thinking 35%
  68. 68 Grok 4.5 35%
  69. 69 Llama 4 Scout 34.8%
  70. 70 Inkling-Small 32.7%
  71. 71 Claude Haiku 4.5 thinking 31.8%
  72. 72 Claude Opus 4.1 non-thinking 31.4%
  73. 73 Grok 4.1 Fast reasoning 31.3%
  74. 74 Gemini 2.5 Flash-Lite Sep 2025, thinking 30.8%
  75. 75 Qwen 3.5 Plus (hosted 397B-A17B) thinking 30.4%
  76. 76 GPT-5 nano 30.4%
  77. 77 Grok 4 Fast reasoning 29.8%
  78. 78 Grok 4.6 28.9%
  79. 79 Grok 4 25.1%
  80. 80 Mistral Large 3 24.6%
  81. 81 Grok 4.3 19.7%
#ModelScoreSettingDateSource
1 Claude Opus 4.7 56.1% Apr 2026 Official leaderboard
2 Gemma 4 31B IT 55% Apr 2026 Official leaderboard
3 Claude Opus 4.8 54.8% May 2026 Official leaderboard
4 Gemini 4 Argon 53.6% Sep 2026 Official leaderboard
5 Kimi K3 52.8% Jul 2026 Official leaderboard
6 GPT-5.6 Sol 52.6% Jul 2026 Official leaderboard
7 Qwen 3.8 27B 52.4% Aug 2026 Official leaderboard
8 Claude Opus 4.5 52.1% thinking Nov 2025 Official leaderboard
9 Claude Fable 5 51.9% Jun 2026 Official leaderboard
10 Gemini 3 Flash 51.9% Dec 2025 Official leaderboard
11 Claude Sonnet 5.5 51.8% Sep 2026 Official leaderboard
12 Claude Opus 4.6 51.6% thinking Feb 2026 Official leaderboard
13 GPT-5.5 51.5% Apr 2026 Official leaderboard
14 Qwen 3.8 Max 51.3% Aug 2026 Official leaderboard
15 GPT-5.4 Mini 50.8% Mar 2026 Official leaderboard
16 MiniMax-M3 50.6% Jun 2026 Official leaderboard
17 Kimi K2.6 50.2% Apr 2026 Official leaderboard
18 Gemini 3.5 Flash 49.9% May 2026 Official leaderboard
19 Kimi K2.5 49.9% thinking Jan 2026 Official leaderboard
20 Gemini 3.1 Flash-Lite 49.5% Mar 2026 Official leaderboard
21 Claude Opus 5 49.4% Jul 2026 Official leaderboard
22 GPT-5.2 49.3% Dec 2025 Official leaderboard
23 Gemini 3.7 Flash 49.2% Aug 2026 Official leaderboard
24 Claude Sonnet 5 48.9% Jun 2026 Official leaderboard
25 Gemini 3.6 Flash 48.7% Jul 2026 Official leaderboard
26 Gemini 3.1 Pro 48.7% Feb 2026 Official leaderboard
27 Claude Fable 5.1 48.5% Sep 2026 Official leaderboard
28 GPT-6 Luna 48.1% Sep 2026 Official leaderboard
29 DeepSeek V4.1 Flash 47.9% Sep 2026 Official leaderboard
30 Muse Spark 1.2 47.7% Aug 2026 Official leaderboard
31 Gemini 3 Pro 47.6% Nov 2025 Official leaderboard
32 Gemini 3.5 Flash-Lite 47.3% Jul 2026 Official leaderboard
33 GPT-5.6 Terra 47% Jul 2026 Official leaderboard
34 Claude Sonnet 4.6 46.6% Feb 2026 Official leaderboard
35 GPT-6.1 Sol 46.5% Sep 2026 Official leaderboard
36 GPT-6 Astra 46.4% Sep 2026 Official leaderboard
37 Claude Opus 5.5 45.8% Sep 2026 Official leaderboard
38 Claude Haiku 5.5 45.8% Oct 2026 Official leaderboard
39 Qwen3.6 27B 45.6% Apr 2026 Official leaderboard
40 Muse Spark 1.1 45.6% Jul 2026 Official leaderboard
41 MiMo-V2.6 Pro 45% Sep 2026 Official leaderboard
42 Qwen 3.6 Plus 44.9% Mar 2026 Official leaderboard
43 GPT-6 Sol 44.8% Sep 2026 Official leaderboard
44 Gemini 2.5 Flash Jun 2025 44.8% non-thinking Jun 2025 Official leaderboard
45 GPT-5.6 Luna 44.2% Jul 2026 Official leaderboard
46 Gemini 2.5 Flash Sep 2025 44.1% non-thinking Sep 2025 Official leaderboard
47 GPT-5 43.7% Aug 2025 Official leaderboard
48 MiMo-V2.6 Flash 43.5% Sep 2026 Official leaderboard
49 GPT-5.4 43.3% xhigh Mar 2026 Official leaderboard
50 MiMo-V2.5 43.3% Apr 2026 Official leaderboard
51 GPT-5.1 43.2% Nov 2025 Official leaderboard
52 GPT-5 mini 43% Aug 2025 Official leaderboard
53 Qwen 3.5 Flash (hosted 35B-A3B) 42.5% Feb 2026 Official leaderboard
54 Qwen3-VL-Plus 42.1% Sep 2025 Official leaderboard
55 Gemini 2.5 Pro Jun 2025 41.9% Jun 2025 Official leaderboard
56 o3 41.8% Apr 2025 Official leaderboard
57 o4-mini 41.1% Apr 2025 Official leaderboard
58 Grok 4.7 40.8% Sep 2026 Official leaderboard
59 Qwen3.7-Plus 39.3% Jun 2026 Official leaderboard
60 Grok 4.20 38.2% reasoning Feb 2026 Official leaderboard
61 GPT-5.4 Nano 38.1% Mar 2026 Official leaderboard
62 Mistral Medium 3.5 37.6% Apr 2026 Official leaderboard
63 Inkling 36.5% Jul 2026 Official leaderboard
64 Claude Sonnet 4.5 36.1% thinking Sep 2025 Official leaderboard
65 Llama 4 Maverick 35.7% Apr 2025 Official leaderboard
66 Gemini 3.8 Flash 35.1% Sep 2026 Official leaderboard
67 Claude Sonnet 4 35% non-thinking May 2025 Official leaderboard
68 Grok 4.5 35% Jul 2026 Official leaderboard
69 Llama 4 Scout 34.8% Apr 2025 Official leaderboard
70 Inkling-Small 32.7% Jul 2026 Official leaderboard
71 Claude Haiku 4.5 31.8% thinking Oct 2025 Official leaderboard
72 Claude Opus 4.1 31.4% non-thinking Aug 2025 Official leaderboard
73 Grok 4.1 Fast 31.3% reasoning Nov 2025 Official leaderboard
74 Gemini 2.5 Flash-Lite Sep 2025 30.8% thinking Sep 2025 Official leaderboard
75 Qwen 3.5 Plus (hosted 397B-A17B) 30.4% thinking Feb 2026 Official leaderboard
76 GPT-5 nano 30.4% Aug 2025 Official leaderboard
77 Grok 4 Fast 29.8% reasoning Sep 2025 Official leaderboard
78 Grok 4.6 28.9% Aug 2026 Official leaderboard
79 Grok 4 25.1% Jul 2025 Official leaderboard
80 Mistral Large 3 24.6% Dec 2025 Official leaderboard
81 Grok 4.3 19.7% Apr 2026 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 4.5
non-thinking thinking
52.1%
Gemini 2.5 Flash
thinking non-thinking
44.8%
Gemini 2.5 Flash
thinking non-thinking
44.1%
Claude Sonnet 4.5
non-thinking
36.1%
Claude Sonnet 4
thinking
35%
Claude Opus 4.1
thinking
31.4%
Grok 4.1 Fast
non-reasoni… reasoning
31.3%
Gemini 2.5 Flash-Lite
non-thinking
30.8%
Grok 4 Fast
non-reasoni… reasoning
29.8%
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 4.7 56.1%
Gemma 4 31B IT 55%
Claude Opus 4.8 54.8%
Gemini 4 Argon 53.6%
Kimi K3 52.8%
GPT-5.6 Sol 52.6%
Qwen 3.8 27B 52.4%
Claude Opus 4.5 thinking 52.1%
Claude Fable 5 51.9%
Gemini 3 Flash 51.9%
Claude Sonnet 5.5 51.8%
Claude Opus 4.6 thinking 51.6%
GPT-5.5 51.5%
Qwen 3.8 Max 51.3%
GPT-5.4 Mini 50.8%
MiniMax-M3 50.6%
Kimi K2.6 50.2%
Gemini 3.5 Flash 49.9%
Kimi K2.5 thinking 49.9%
Gemini 3.1 Flash-Lite 49.5%
Claude Opus 5 49.4%
GPT-5.2 49.3%
Gemini 3.7 Flash 49.2%
Claude Sonnet 5 48.9%
Gemini 3.6 Flash 48.7%
Gemini 3.1 Pro 48.7%
Claude Fable 5.1 48.5%
GPT-6 Luna 48.1%
DeepSeek V4.1 Flash 47.9%
Muse Spark 1.2 47.7%
Gemini 3 Pro 47.6%
Gemini 3.5 Flash-Lite 47.3%
GPT-5.6 Terra 47%
Claude Sonnet 4.6 46.6%
GPT-6.1 Sol 46.5%
GPT-6 Astra 46.4%
Claude Opus 5.5 45.8%
Claude Haiku 5.5 45.8%
Qwen3.6 27B 45.6%
Muse Spark 1.1 45.6%
MiMo-V2.6 Pro 45%
Claude Opus 4.5 non-thinking 45%
Qwen 3.6 Plus 44.9%
GPT-6 Sol 44.8%
Gemini 2.5 Flash Jun 2025 non-thinking 44.8%
GPT-5.6 Luna 44.2%
Gemini 2.5 Flash Sep 2025 non-thinking 44.1%
GPT-5 43.7%
MiMo-V2.6 Flash 43.5%
GPT-5.4 xhigh 43.3%
MiMo-V2.5 43.3%
GPT-5.1 43.2%
GPT-5 mini 43%
Qwen 3.5 Flash (hosted 35B-A3B) 42.5%
Qwen3-VL-Plus 42.1%
Gemini 2.5 Pro Jun 2025 41.9%
o3 41.8%
o4-mini 41.1%
Grok 4.7 40.8%
Gemini 2.5 Flash Jun 2025 thinking 39.6%
Gemini 2.5 Flash Sep 2025 thinking 39.4%
Qwen3.7-Plus 39.3%
Grok 4.20 reasoning 38.2%
GPT-5.4 Nano 38.1%
Mistral Medium 3.5 37.6%
Inkling 36.5%
Claude Sonnet 4.5 thinking 36.1%
Llama 4 Maverick 35.7%
Gemini 3.8 Flash 35.1%
Claude Sonnet 4 non-thinking 35%
Grok 4.5 35%
Llama 4 Scout 34.8%
Claude Sonnet 4.5 non-thinking 32.9%
Inkling-Small 32.7%
Claude Sonnet 4 thinking 32%
Claude Haiku 4.5 thinking 31.8%
Claude Opus 4.1 non-thinking 31.4%
Grok 4.1 Fast reasoning 31.3%
Gemini 2.5 Flash-Lite Sep 2025 thinking 30.8%
Qwen 3.5 Plus (hosted 397B-A17B) thinking 30.4%
Claude Opus 4.1 thinking 30.4%
GPT-5 nano 30.4%
Grok 4 Fast reasoning 29.8%
Grok 4.6 28.9%
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 28%
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 27.5%
Grok 4 25.1%
Mistral Large 3 24.6%
Grok 4.3 19.7%
Grok 4 Fast non-reasoning 16%
Grok 4.1 Fast non-reasoning 11.9%

5 models have held number one since its release in Oct 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Anthropic
Own leaderboard 5 leaders
Claude Opus 4.7 56.1%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 4.7 Claude Opus 4.5 Gemini 3 Pro Gemini 2.5 Flash Llama 4 Maverick 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 4.7 Gemini 3 Pro

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Google DeepMind Anthropic
SAGE v1.1 (Vals) Passed 85% in Oct 2026, by Claude Opus 5.5
Claude Opus 5.5 91.3%
SAGE (Vals) 5 leaders
Claude Opus 4.7 56.1%
SAGE v1.1 (Vals) Passed 85% in Oct 2026, by Claude Opus 5.5
Claude Opus 5.5 91.3%
Replaced by SAGE v1.1 (Vals)

More in Other

All 41 in Other
  • Personal Superintelligence Index (Meta) In use Meta's composite of seven personal and knowledge-work evals: Agents' Last Exam, AutomationBench-AA, FocusBench, JobBench, GDP.pdf, VBT and DRACO.
  • Artificial Analysis Intelligence Index v5 In use Version 5 of Artificial Analysis's index: nine evaluations in four equal categories, most with private test sets, run three times each in its own agent harness. Gemini 4 Carbon 59.0
  • Anthropic ECI (AECI) Only in system cards Index that combines a basket of capability benchmarks into one capability score to track trends across Anthropic models. Claude Opus 5.5 169.4
  • Qwen in-house English automatic evaluation Only in system cards Models answer Qwen in-house English datasets on knowledge, comprehension, coding, math and related skills. Skills and weights change between reports. Qwen2-72B 72.9%
  • Recognition of model-written user turns (Anthropic) Only in system cards Models write synthetic user turns for real Claude Code transcripts, then judge which version is real. Score is accuracy above chance. Claude Mythos Preview 0.35
  • LiveBench Dropped Models answer monthly refreshed questions across math, coding, reasoning, language, instruction following, and data analysis. Scored automatically against objective ground truth. Claude Fable 5.1 85.7%