Benchmark Almanac

MortgageTax

Claude Opus 5 72.1% leads since Jul 2026 Not in model cards

Vision models read mortgage tax certificates given as images. They extract the year, parcel number and county, and compute the annualized amount due.

1,258 documents with a 20-sample public validation set. Vals archived it as saturated by 2026-09-01.

Made by Vals AI (data from Vontive) Finance Tasks kept private Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
Leaderboard
86
Models scored on its own leaderboard

Claude Opus 5 leads with 72.1% among models released by Sep 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in Sep 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

86 models. Scroll sideways for the rest.

  1. 72.1 Claude Opus 5
  2. 70.8 Claude Fable 5.1
  3. 70.3 Claude Opus 4.7
  4. 70.0 Claude Sonnet 5
  5. 69.9 Claude Opus 4.8
  6. 69.4 Gemini 3.1 Pro
  7. 69.1 Gemini 3 Pro
  8. 68.9 Gemini 2.5 Pro
  9. 68.9 Claude Fable 5
  10. 68.8 GPT-5.5
  11. 68.7 Gemini 3 Flash
  12. 68.7 Claude 3.7 Sonnet
  13. 68.7 Claude Opus 4.5
  14. 68.7 Gemini 3.5 Flash-Lite
  15. 68.5 Claude Opus 4.6
  16. 68.4 MiniMax-M3
  17. 68.3 GPT-5.4
  18. 68.3 Qwen3.6 27B
  19. 68.1 Gemini 3.5 Flash
  20. 68.0 Gemini 3.1 Flash-Lite
  21. 68.0 Qwen 3.6 Plus
  22. 67.7 Claude Sonnet 4.6
  23. 67.6 Gemini 3.6 Flash
  24. 67.4 Qwen 3.5 Flash (hosted 35B-A3B)
  25. 67.3 GPT-5.6 Terra
  26. 67.3 GPT-5.6 Luna
  27. 67.3 GPT-5.6 Sol
  28. 67.1 GPT-5.2
  29. 66.9 GPT-5 mini
  30. 66.7 Gemini 3.7 Flash
  31. 66.5 Kimi K2.5
  32. 66.3 Kimi K3
  33. 66.2 Qwen3.7-Plus
  34. 66.1 Muse Spark 1.1
  35. 65.9 GPT-4.1
  36. 65.8 Kimi K2.6
  37. 65.7 o3
  38. 65.5 GPT-4.1 mini
  39. 65.5 GPT-5
  40. 65.4 Muse Spark 1.2
  41. 65.3 Gemini 3.8 Flash
  42. 64.9 Qwen 3.8 27B
  43. 64.8 o4-mini
  44. 64.2 Grok 4.6
  45. 64.1 Claude 3.5 Sonnet
  46. 64.0 Claude Sonnet 4.5
  47. 64.0 Inkling
  48. 64.0 Qwen 3.8 Max
  49. 63.5 GPT-5.4 Mini
  50. 62.6 Gemini 2.5 Flash
  51. 62.5 Claude Sonnet 4
  52. 62.3 Inkling-Small
  53. 62.2 Claude Haiku 4.5
  54. 62.1 Magistral Small 1.2
  55. 61.8 Grok 4.5
  56. 61.4 GPT-5.1
  57. 61.4 Gemma 4 31B IT
  58. 61.2 Mistral Small 3.1
  59. 61.1 Claude Opus 4.1
  60. 61.0 GPT-4o
  61. 60.8 Qwen 3.5 Plus (hosted 397B-A17B)
  62. 60.4 Gemini 2.0 Flash
  63. 59.3 MiMo-V2.5
  64. 59.1 GPT-5.4 Nano
  65. 58.6 Claude Opus 4
  66. 58.5 Llama 4 Maverick
  67. 58.4 Gemini 2.5 Flash
  68. 57.8 Llama 4 Scout
  69. 57.5 Gemini 2.5 Flash-Lite
  70. 56.8 Gemini 1.5 Pro
  71. 54.5 GPT-4o mini
  72. 53.6 GPT-5 nano
  73. 52.8 GPT-4.1 nano
  74. 52.1 Mistral Large 3
  75. 51.9 Magistral Medium 1.2
  76. 48.3 Grok 4.3
  77. 45.4 Grok 4.20
  78. 44.5 Grok 4
  79. 42.8 Gemini 1.5 Flash
  80. 42.6 Grok 4.1 Fast
  81. 42.1 Grok 4 Fast
  82. 38.8 Llama 3.2 90B
  83. 36.5 Mistral Medium 3.1
  84. 28.9 Mistral Medium 3.5
  85. 23.3 Llama 3.2 11B
  86. 3.46 Grok 2 Vision (1212)
  1. 1 Claude Opus 5 72.1%
  2. 2 Claude Fable 5.1 70.8%
  3. 3 Claude Opus 4.7 70.3%
  4. 4 Claude Sonnet 5 70%
  5. 5 Claude Opus 4.8 69.9%
  6. 6 Gemini 3.1 Pro 69.4%
  7. 7 Gemini 3 Pro 69.1%
  8. 8 Gemini 2.5 Pro Jun 2025 68.9%
  9. 9 Claude Fable 5 68.9%
  10. 10 GPT-5.5 68.8%
  11. 11 Gemini 3 Flash 68.7%
  12. 12 Claude 3.7 Sonnet non-thinking 68.7%
  13. 13 Claude Opus 4.5 non-thinking 68.7%
  14. 14 Gemini 3.5 Flash-Lite 68.7%
  15. 15 Claude Opus 4.6 thinking 68.5%
  16. 16 MiniMax-M3 68.4%
  17. 17 GPT-5.4 xhigh 68.3%
  18. 18 Qwen3.6 27B 68.3%
  19. 19 Gemini 3.5 Flash 68.1%
  20. 20 Gemini 3.1 Flash-Lite 68%
  21. 21 Qwen 3.6 Plus 68%
  22. 22 Claude Sonnet 4.6 67.7%
  23. 23 Gemini 3.6 Flash 67.6%
  24. 24 Qwen 3.5 Flash (hosted 35B-A3B) 67.4%
  25. 25 GPT-5.6 Terra 67.3%
  26. 26 GPT-5.6 Luna 67.3%
  27. 27 GPT-5.6 Sol 67.3%
  28. 28 GPT-5.2 67.1%
  29. 29 GPT-5 mini 66.9%
  30. 30 Gemini 3.7 Flash 66.7%
  31. 31 Kimi K2.5 thinking 66.5%
  32. 32 Kimi K3 66.3%
  33. 33 Qwen3.7-Plus 66.2%
  34. 34 Muse Spark 1.1 66.1%
  35. 35 GPT-4.1 65.9%
  36. 36 Kimi K2.6 65.8%
  37. 37 o3 65.7%
  38. 38 GPT-4.1 mini 65.5%
  39. 39 GPT-5 65.5%
  40. 40 Muse Spark 1.2 65.4%
  41. 41 Gemini 3.8 Flash 65.3%
  42. 42 Qwen 3.8 27B 64.9%
  43. 43 o4-mini 64.8%
  44. 44 Grok 4.6 64.2%
  45. 45 Claude 3.5 Sonnet 64.1%
  46. 46 Claude Sonnet 4.5 thinking 64%
  47. 47 Inkling 64%
  48. 48 Qwen 3.8 Max 64%
  49. 49 GPT-5.4 Mini 63.5%
  50. 50 Gemini 2.5 Flash Sep 2025, non-thinking 62.6%
  51. 51 Claude Sonnet 4 non-thinking 62.5%
  52. 52 Inkling-Small 62.3%
  53. 53 Claude Haiku 4.5 thinking 62.2%
  54. 54 Magistral Small 1.2 62.1%
  55. 55 Grok 4.5 61.8%
  56. 56 GPT-5.1 61.4%
  57. 57 Gemma 4 31B IT 61.4%
  58. 58 Mistral Small 3.1 61.2%
  59. 59 Claude Opus 4.1 non-thinking 61.1%
  60. 60 GPT-4o Aug 2024 61%
  61. 61 Qwen 3.5 Plus (hosted 397B-A17B) thinking 60.8%
  62. 62 Gemini 2.0 Flash Feb 2025 60.4%
  63. 63 MiMo-V2.5 59.3%
  64. 64 GPT-5.4 Nano 59.1%
  65. 65 Claude Opus 4 non-thinking 58.6%
  66. 66 Llama 4 Maverick 58.5%
  67. 67 Gemini 2.5 Flash Apr 2025, non-thinking 58.4%
  68. 68 Llama 4 Scout 57.8%
  69. 69 Gemini 2.5 Flash-Lite Sep 2025, thinking 57.5%
  70. 70 Gemini 1.5 Pro Sep 2024 56.8%
  71. 71 GPT-4o mini 54.5%
  72. 72 GPT-5 nano 53.6%
  73. 73 GPT-4.1 nano 52.8%
  74. 74 Mistral Large 3 52.1%
  75. 75 Magistral Medium 1.2 51.9%
  76. 76 Grok 4.3 48.3%
  77. 77 Grok 4.20 reasoning 45.4%
  78. 78 Grok 4 44.5%
  79. 79 Gemini 1.5 Flash Sep 2024 42.8%
  80. 80 Grok 4.1 Fast reasoning 42.6%
  81. 81 Grok 4 Fast reasoning 42.1%
  82. 82 Llama 3.2 90B 38.8%
  83. 83 Mistral Medium 3.1 36.5%
  84. 84 Mistral Medium 3.5 28.9%
  85. 85 Llama 3.2 11B 23.3%
  86. 86 Grok 2 Vision (1212) 3.46%
#ModelScoreSettingDateSource
1 Claude Opus 5 72.1% Jul 2026 Official leaderboard
2 Claude Fable 5.1 70.8% Sep 2026 Official leaderboard
3 Claude Opus 4.7 70.3% Apr 2026 Official leaderboard
4 Claude Sonnet 5 70% Jun 2026 Official leaderboard
5 Claude Opus 4.8 69.9% May 2026 Official leaderboard
6 Gemini 3.1 Pro 69.4% Feb 2026 Official leaderboard
7 Gemini 3 Pro 69.1% Nov 2025 Official leaderboard
8 Gemini 2.5 Pro Jun 2025 68.9% Jun 2025 Official leaderboard
9 Claude Fable 5 68.9% Jun 2026 Official leaderboard
10 GPT-5.5 68.8% Apr 2026 Official leaderboard
11 Gemini 3 Flash 68.7% Dec 2025 Official leaderboard
12 Claude 3.7 Sonnet 68.7% non-thinking Feb 2025 Official leaderboard
13 Claude Opus 4.5 68.7% non-thinking Nov 2025 Official leaderboard
14 Gemini 3.5 Flash-Lite 68.7% Jul 2026 Official leaderboard
15 Claude Opus 4.6 68.5% thinking Feb 2026 Official leaderboard
16 MiniMax-M3 68.4% Jun 2026 Official leaderboard
17 GPT-5.4 68.3% xhigh Mar 2026 Official leaderboard
18 Qwen3.6 27B 68.3% Apr 2026 Official leaderboard
19 Gemini 3.5 Flash 68.1% May 2026 Official leaderboard
20 Gemini 3.1 Flash-Lite 68% Mar 2026 Official leaderboard
21 Qwen 3.6 Plus 68% Mar 2026 Official leaderboard
22 Claude Sonnet 4.6 67.7% Feb 2026 Official leaderboard
23 Gemini 3.6 Flash 67.6% Jul 2026 Official leaderboard
24 Qwen 3.5 Flash (hosted 35B-A3B) 67.4% Feb 2026 Official leaderboard
25 GPT-5.6 Terra 67.3% Jul 2026 Official leaderboard
26 GPT-5.6 Luna 67.3% Jul 2026 Official leaderboard
27 GPT-5.6 Sol 67.3% Jul 2026 Official leaderboard
28 GPT-5.2 67.1% Dec 2025 Official leaderboard
29 GPT-5 mini 66.9% Aug 2025 Official leaderboard
30 Gemini 3.7 Flash 66.7% Aug 2026 Official leaderboard
31 Kimi K2.5 66.5% thinking Jan 2026 Official leaderboard
32 Kimi K3 66.3% Jul 2026 Official leaderboard
33 Qwen3.7-Plus 66.2% Jun 2026 Official leaderboard
34 Muse Spark 1.1 66.1% Jul 2026 Official leaderboard
35 GPT-4.1 65.9% Apr 2025 Official leaderboard
36 Kimi K2.6 65.8% Apr 2026 Official leaderboard
37 o3 65.7% Apr 2025 Official leaderboard
38 GPT-4.1 mini 65.5% Apr 2025 Official leaderboard
39 GPT-5 65.5% Aug 2025 Official leaderboard
40 Muse Spark 1.2 65.4% Aug 2026 Official leaderboard
41 Gemini 3.8 Flash 65.3% Sep 2026 Official leaderboard
42 Qwen 3.8 27B 64.9% Aug 2026 Official leaderboard
43 o4-mini 64.8% Apr 2025 Official leaderboard
44 Grok 4.6 64.2% Aug 2026 Official leaderboard
45 Claude 3.5 Sonnet 64.1% Jun 2024 Official leaderboard
46 Claude Sonnet 4.5 64% thinking Sep 2025 Official leaderboard
47 Inkling 64% Jul 2026 Official leaderboard
48 Qwen 3.8 Max 64% Aug 2026 Official leaderboard
49 GPT-5.4 Mini 63.5% Mar 2026 Official leaderboard
50 Gemini 2.5 Flash Sep 2025 62.6% non-thinking Sep 2025 Official leaderboard
51 Claude Sonnet 4 62.5% non-thinking May 2025 Official leaderboard
52 Inkling-Small 62.3% Jul 2026 Official leaderboard
53 Claude Haiku 4.5 62.2% thinking Oct 2025 Official leaderboard
54 Magistral Small 1.2 62.1% Sep 2025 Official leaderboard
55 Grok 4.5 61.8% Jul 2026 Official leaderboard
56 GPT-5.1 61.4% Nov 2025 Official leaderboard
57 Gemma 4 31B IT 61.4% Apr 2026 Official leaderboard
58 Mistral Small 3.1 61.2% Mar 2025 Official leaderboard
59 Claude Opus 4.1 61.1% non-thinking Aug 2025 Official leaderboard
60 GPT-4o Aug 2024 61% Aug 2024 Official leaderboard
61 Qwen 3.5 Plus (hosted 397B-A17B) 60.8% thinking Feb 2026 Official leaderboard
62 Gemini 2.0 Flash Feb 2025 60.4% Feb 2025 Official leaderboard
63 MiMo-V2.5 59.3% Apr 2026 Official leaderboard
64 GPT-5.4 Nano 59.1% Mar 2026 Official leaderboard
65 Claude Opus 4 58.6% non-thinking May 2025 Official leaderboard
66 Llama 4 Maverick 58.5% Apr 2025 Official leaderboard
67 Gemini 2.5 Flash Apr 2025 58.4% non-thinking Apr 2025 Official leaderboard
68 Llama 4 Scout 57.8% Apr 2025 Official leaderboard
69 Gemini 2.5 Flash-Lite Sep 2025 57.5% thinking Sep 2025 Official leaderboard
70 Gemini 1.5 Pro Sep 2024 56.8% Sep 2024 Official leaderboard
71 GPT-4o mini 54.5% Jul 2024 Official leaderboard
72 GPT-5 nano 53.6% Aug 2025 Official leaderboard
73 GPT-4.1 nano 52.8% Apr 2025 Official leaderboard
74 Mistral Large 3 52.1% Dec 2025 Official leaderboard
75 Magistral Medium 1.2 51.9% Sep 2025 Official leaderboard
76 Grok 4.3 48.3% Apr 2026 Official leaderboard
77 Grok 4.20 45.4% reasoning Feb 2026 Official leaderboard
78 Grok 4 44.5% Jul 2025 Official leaderboard
79 Gemini 1.5 Flash Sep 2024 42.8% Aug 2024 Official leaderboard
80 Grok 4.1 Fast 42.6% reasoning Nov 2025 Official leaderboard
81 Grok 4 Fast 42.1% reasoning Sep 2025 Official leaderboard
82 Llama 3.2 90B 38.8% Sep 2024 Official leaderboard
83 Mistral Medium 3.1 36.5% Aug 2025 Official leaderboard
84 Mistral Medium 3.5 28.9% Apr 2026 Official leaderboard
85 Llama 3.2 11B 23.3% Sep 2024 Official leaderboard
86 Grok 2 Vision (1212) 3.46% Dec 2024 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude 3.7 Sonnet
thinking
68.7%
Claude Opus 4.5
thinking
68.7%
Gemini 2.5 Flash
thinking
62.6%
Claude Sonnet 4
thinking non-thinking
62.5%
Claude Opus 4.1
thinking non-thinking
61.1%
Gemini 2.5 Flash
thinking
58.4%
Gemini 2.5 Flash-Lite
non-thinking
57.5%
Grok 4.1 Fast
non-reasoni… reasoning
42.6%
Grok 4 Fast
non-reasoni… reasoning
42.1%
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5 72.1%
Claude Fable 5.1 70.8%
Claude Opus 4.7 70.3%
Claude Sonnet 5 70%
Claude Opus 4.8 69.9%
Gemini 3.1 Pro 69.4%
Gemini 3 Pro 69.1%
Gemini 2.5 Pro Jun 2025 68.9%
Claude Fable 5 68.9%
GPT-5.5 68.8%
Gemini 3 Flash 68.7%
Claude 3.7 Sonnet non-thinking 68.7%
Claude Opus 4.5 non-thinking 68.7%
Gemini 3.5 Flash-Lite 68.7%
Claude Opus 4.6 thinking 68.5%
MiniMax-M3 68.4%
GPT-5.4 xhigh 68.3%
Qwen3.6 27B 68.3%
Gemini 3.5 Flash 68.1%
Gemini 3.1 Flash-Lite 68%
Qwen 3.6 Plus 68%
Claude Sonnet 4.6 67.7%
Claude Opus 4.5 thinking 67.7%
Gemini 3.6 Flash 67.6%
Qwen 3.5 Flash (hosted 35B-A3B) 67.4%
GPT-5.6 Terra 67.3%
GPT-5.6 Luna 67.3%
GPT-5.6 Sol 67.3%
Gemini 2.5 Pro Jun 2025 67.2%
GPT-5.2 67.1%
GPT-5 mini 66.9%
Claude 3.7 Sonnet thinking 66.8%
Gemini 3.7 Flash 66.7%
Kimi K2.5 thinking 66.5%
Kimi K3 66.3%
Qwen3.7-Plus 66.2%
Muse Spark 1.1 66.1%
GPT-4.1 65.9%
Kimi K2.6 65.8%
o3 65.7%
GPT-4.1 mini 65.5%
GPT-5 65.5%
Muse Spark 1.2 65.4%
Gemini 3.8 Flash 65.3%
Qwen 3.8 27B 64.9%
o4-mini 64.8%
Grok 4.6 64.2%
Claude 3.5 Sonnet 64.1%
Claude Sonnet 4.5 thinking 64%
Inkling 64%
Qwen 3.8 Max 64%
GPT-5.4 Mini 63.5%
Gemini 2.5 Flash Sep 2025 non-thinking 62.6%
Claude Sonnet 4 non-thinking 62.5%
Inkling-Small 62.3%
Claude Haiku 4.5 thinking 62.2%
Magistral Small 1.2 62.1%
Gemini 2.5 Flash Sep 2025 thinking 61.9%
Grok 4.5 61.8%
GPT-5.1 61.4%
Gemma 4 31B IT 61.4%
Mistral Small 3.1 61.2%
Claude Opus 4.1 non-thinking 61.1%
GPT-4o Aug 2024 61%
Qwen 3.5 Plus (hosted 397B-A17B) thinking 60.8%
Gemini 2.0 Flash Feb 2025 60.4%
Gemini 2.0 Flash Feb 2025 59.7%
MiMo-V2.5 59.3%
GPT-5.4 Nano 59.1%
Claude Opus 4 non-thinking 58.6%
Llama 4 Maverick 58.5%
Gemini 2.5 Flash Apr 2025 non-thinking 58.4%
Llama 4 Scout 57.8%
Gemini 2.5 Flash-Lite Sep 2025 thinking 57.5%
Gemini 2.5 Flash Apr 2025 thinking 57.5%
GPT-4o Aug 2024 57.4%
Gemini 1.5 Pro Sep 2024 56.8%
Claude Opus 4.1 thinking 56.1%
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 55.8%
GPT-4o mini 54.5%
GPT-5 nano 53.6%
GPT-4.1 nano 52.8%
Mistral Large 3 52.1%
Magistral Medium 1.2 51.9%
Claude Sonnet 4 thinking 49.9%
Grok 4.3 48.3%
Grok 4.20 reasoning 45.4%
Grok 4 44.5%
Gemini 1.5 Flash Sep 2024 42.8%
Grok 4.1 Fast reasoning 42.6%
Grok 4 Fast reasoning 42.1%
Llama 3.2 90B 38.8%
Mistral Medium 3.1 36.5%
Grok 4.1 Fast non-reasoning 34.3%
Mistral Medium 3.5 28.9%
Grok 4 Fast non-reasoning 25.3%
Llama 3.2 11B 23.3%
Grok 2 Vision (1212) 3.46%

7 models have held number one since 2023

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic Google DeepMind
Own leaderboard Not tested since Sep 2026
Claude Opus 5 72.1%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Opus 5 Gemini 3.1 Pro Gemini 2.5 Pro Claude 3.5 Sonnet 0% 20% 40% 60% 80% 100% 2024 2025 2026 Near the ceiling (85%) Claude Opus 5 Claude 3.5 Sonnet

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 140 in Law, finance and medicine
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%
  • PRBench In use Model answers 1,100 expert-written law and finance questions. LLM judge grades against weighted expert rubrics of 10-30 criteria each. Muse Spark 1.3 61.6%