Benchmark Almanac

MedScribe

Claude Opus 5.5 91.4% leads since Sep 2026 Not in model cards

Models write SOAP notes from clinical visit transcripts. Notes are graded against expert gold-standard rubrics for required content and section structure.

Transcripts are synthetic, generated with NoteChat from real de-identified notes. Experts could not tell them from real ones. 109 models on 2026-10-29.

Made by Vals AI (data partner Protege) Released Feb 2026 Medicine Tasks kept private Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
Leaderboard
98
Models scored on its own leaderboard

Claude Opus 5.5 leads with 91.4%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

98 models. Scroll sideways for the rest.

  1. 91.4 Claude Opus 5.5
  2. 91.3 Claude Fable 5.1
  3. 91.1 Claude Sonnet 5.5
  4. 91.0 Claude Opus 5
  5. 90.1 Muse Spark 1.2
  6. 89.5 Claude Haiku 5.5
  7. 89.4 Grok 4.7
  8. 89.3 Muse Wonder Full
  9. 88.9 GLM-5.3-Flash
  10. 88.9 Muse Spark 1.1
  11. 88.8 GLM-5.3
  12. 88.5 Claude Fable 5
  13. 88.3 MiMo-V2.6 Pro
  14. 88.1 GPT-5.1
  15. 88.0 Kimi K3
  16. 87.9 GPT-6 Astra
  17. 87.4 Gemini 4 Argon
  18. 87.3 MiniMax-M3
  19. 86.9 Grok 4.5
  20. 86.9 GPT-5.5
  21. 86.7 Claude Opus 4.6
  22. 86.5 Grok 4.6
  23. 86.5 GPT-6.1 Sol
  24. 85.9 Muse Spark
  25. 85.8 Claude Opus 4.8
  26. 85.5 DeepSeek V4.1 Flash
  27. 85.4 Inkling
  28. 85.3 Claude Opus 4.5
  29. 85.3 MiMo-V2.6 Flash
  30. 85.2 Claude Haiku 4.5
  31. 85.2 GPT-5.6 Sol
  32. 85.0 Qwen 3.8 Max
  33. 84.5 Claude Sonnet 4.5
  34. 84.5 Gemini 3.8 Flash
  35. 84.4 GPT-5.2
  36. 84.4 GPT-5.6 Luna
  37. 84.1 Inkling-Small
  38. 83.9 Gemini 3.7 Flash
  39. 83.8 Qwen 3.8 27B
  40. 83.7 MiMo-V2.5-Pro
  41. 83.7 GPT-6 Luna
  42. 83.7 GPT-5
  43. 83.6 Tencent Hy4 preview
  44. 83.5 GLM-5.2
  45. 83.0 Gemini 2.5 Flash
  46. 83.0 Claude Opus 4.7
  47. 82.9 GPT-5.6 Terra
  48. 82.0 GPT-6 Sol
  49. 81.6 Grok 4 Fast
  50. 80.9 Ling-3.0-flash
  51. 80.8 MiniMax-M2.1
  52. 80.6 GPT-5 mini
  53. 80.4 Mistral Large 4
  54. 80.4 DeepSeek V4 Flash 0731
  55. 80.2 DeepSeek V4 Pro 0813
  56. 79.9 MiniMax-M2.7
  57. 79.7 Gemini 3.6 Flash
  58. 79.4 Qwen3.7-Max
  59. 78.7 Grok 4.1 Fast
  60. 78.5 Gemini 2.5 Flash
  61. 78.2 Grok 4
  62. 78.2 Kimi K2.6
  63. 77.5 GPT-5.4
  64. 77.1 Qwen3-VL-Plus
  65. 77.1 GPT-5.4 Nano
  66. 77.0 Qwen 3.6 Plus
  67. 76.7 o3
  68. 76.6 Gemini 3.5 Flash
  69. 76.4 Kimi K2.5
  70. 76.1 Gemini 3.1 Pro
  71. 76.0 Claude Sonnet 5
  72. 75.8 Gemini 2.5 Flash-Lite
  73. 75.6 Ling-3.0-flash-Fin
  74. 75.1 DeepSeek-V4-Pro
  75. 74.4 Grok 4.3
  76. 73.9 Claude Opus 4.1
  77. 73.5 Gemini 2.5 Pro
  78. 72.9 GPT-5 nano
  79. 72.7 Qwen3-Max
  80. 72.4 Claude Sonnet 4
  81. 72.3 GLM-5.1
  82. 72.2 MiMo-V2.5
  83. 72.0 Gemini 3 Pro
  84. 70.9 Gemini 3.5 Flash-Lite
  85. 70.6 Qwen 3.5 Flash (hosted 35B-A3B)
  86. 69.9 Gemini 3 Flash
  87. 69.1 o4-mini
  88. 68.6 GLM-4.7
  89. 67.7 Mistral Medium 3.5
  90. 65.9 Laguna M.1
  91. 63.9 Gemini 3.1 Flash-Lite
  92. 63.4 Grok 4.20
  93. 61.4 Laguna XS.2
  94. 55.7 Command A+
  95. 55.1 Mercury 2.5
  96. 54.2 Llama 4 Maverick
  97. 50.6 Llama 4 Scout
  98. 4.27 Nemotron 3.5 Lightning 30B-A3B
  1. 1 Claude Opus 5.5 91.4%
  2. 2 Claude Fable 5.1 91.3%
  3. 3 Claude Sonnet 5.5 91.1%
  4. 4 Claude Opus 5 91%
  5. 5 Muse Spark 1.2 90.1%
  6. 6 Claude Haiku 5.5 89.5%
  7. 7 Grok 4.7 89.4%
  8. 8 Muse Wonder Full 89.3%
  9. 9 GLM-5.3-Flash 88.9%
  10. 10 Muse Spark 1.1 88.9%
  11. 11 GLM-5.3 88.8%
  12. 12 Claude Fable 5 88.5%
  13. 13 MiMo-V2.6 Pro 88.3%
  14. 14 GPT-5.1 88.1%
  15. 15 Kimi K3 88%
  16. 16 GPT-6 Astra 87.9%
  17. 17 Gemini 4 Argon 87.4%
  18. 18 MiniMax-M3 87.3%
  19. 19 Grok 4.5 86.9%
  20. 20 GPT-5.5 86.9%
  21. 21 Claude Opus 4.6 non-thinking 86.7%
  22. 22 Grok 4.6 86.5%
  23. 23 GPT-6.1 Sol 86.5%
  24. 24 Muse Spark 85.9%
  25. 25 Claude Opus 4.8 85.8%
  26. 26 DeepSeek V4.1 Flash 85.5%
  27. 27 Inkling 85.4%
  28. 28 Claude Opus 4.5 thinking 85.3%
  29. 29 MiMo-V2.6 Flash 85.3%
  30. 30 Claude Haiku 4.5 thinking 85.2%
  31. 31 GPT-5.6 Sol 85.2%
  32. 32 Qwen 3.8 Max 85%
  33. 33 Claude Sonnet 4.5 non-thinking 84.5%
  34. 34 Gemini 3.8 Flash 84.5%
  35. 35 GPT-5.2 84.4%
  36. 36 GPT-5.6 Luna 84.4%
  37. 37 Inkling-Small 84.1%
  38. 38 Gemini 3.7 Flash 83.9%
  39. 39 Qwen 3.8 27B 83.8%
  40. 40 MiMo-V2.5-Pro 83.7%
  41. 41 GPT-6 Luna 83.7%
  42. 42 GPT-5 83.7%
  43. 43 Tencent Hy4 preview 83.6%
  44. 44 GLM-5.2 83.5%
  45. 45 Gemini 2.5 Flash Jun 2025, thinking 83%
  46. 46 Claude Opus 4.7 83%
  47. 47 GPT-5.6 Terra 82.9%
  48. 48 GPT-6 Sol 82%
  49. 49 Grok 4 Fast reasoning 81.6%
  50. 50 Ling-3.0-flash 80.9%
  51. 51 MiniMax-M2.1 80.8%
  52. 52 GPT-5 mini 80.6%
  53. 53 Mistral Large 4 80.4%
  54. 54 DeepSeek V4 Flash 0731 80.4%
  55. 55 DeepSeek V4 Pro 0813 80.2%
  56. 56 MiniMax-M2.7 79.9%
  57. 57 Gemini 3.6 Flash 79.7%
  58. 58 Qwen3.7-Max 79.4%
  59. 59 Grok 4.1 Fast reasoning 78.7%
  60. 60 Gemini 2.5 Flash Sep 2025, thinking 78.5%
  61. 61 Grok 4 78.2%
  62. 62 Kimi K2.6 78.2%
  63. 63 GPT-5.4 xhigh 77.5%
  64. 64 Qwen3-VL-Plus 77.1%
  65. 65 GPT-5.4 Nano 77.1%
  66. 66 Qwen 3.6 Plus 77%
  67. 67 o3 76.7%
  68. 68 Gemini 3.5 Flash 76.6%
  69. 69 Kimi K2.5 thinking 76.4%
  70. 70 Gemini 3.1 Pro 76.1%
  71. 71 Claude Sonnet 5 76%
  72. 72 Gemini 2.5 Flash-Lite Sep 2025, non-thinking 75.8%
  73. 73 Ling-3.0-flash-Fin 75.6%
  74. 74 DeepSeek-V4-Pro 75.1%
  75. 75 Grok 4.3 74.4%
  76. 76 Claude Opus 4.1 thinking 73.9%
  77. 77 Gemini 2.5 Pro Jun 2025 73.5%
  78. 78 GPT-5 nano 72.9%
  79. 79 Qwen3-Max 72.7%
  80. 80 Claude Sonnet 4 non-thinking 72.4%
  81. 81 GLM-5.1 72.3%
  82. 82 MiMo-V2.5 72.2%
  83. 83 Gemini 3 Pro 72%
  84. 84 Gemini 3.5 Flash-Lite 70.9%
  85. 85 Qwen 3.5 Flash (hosted 35B-A3B) 70.6%
  86. 86 Gemini 3 Flash 69.9%
  87. 87 o4-mini 69.1%
  88. 88 GLM-4.7 68.6%
  89. 89 Mistral Medium 3.5 67.7%
  90. 90 Laguna M.1 65.9%
  91. 91 Gemini 3.1 Flash-Lite 63.9%
  92. 92 Grok 4.20 reasoning 63.4%
  93. 93 Laguna XS.2 61.4%
  94. 94 Command A+ 55.7%
  95. 95 Mercury 2.5 55.1%
  96. 96 Llama 4 Maverick 54.2%
  97. 97 Llama 4 Scout 50.6%
  98. 98 Nemotron 3.5 Lightning 30B-A3B 4.27%
#ModelScoreSettingDateSource
1 Claude Opus 5.5 91.4% Sep 2026 Official leaderboard
2 Claude Fable 5.1 91.3% Sep 2026 Official leaderboard
3 Claude Sonnet 5.5 91.1% Sep 2026 Official leaderboard
4 Claude Opus 5 91% Jul 2026 Official leaderboard
5 Muse Spark 1.2 90.1% Aug 2026 Official leaderboard
6 Claude Haiku 5.5 89.5% Oct 2026 Official leaderboard
7 Grok 4.7 89.4% Sep 2026 Official leaderboard
8 Muse Wonder Full 89.3% Oct 2026 Official leaderboard
9 GLM-5.3-Flash 88.9% Aug 2026 Official leaderboard
10 Muse Spark 1.1 88.9% Jul 2026 Official leaderboard
11 GLM-5.3 88.8% Aug 2026 Official leaderboard
12 Claude Fable 5 88.5% Jun 2026 Official leaderboard
13 MiMo-V2.6 Pro 88.3% Sep 2026 Official leaderboard
14 GPT-5.1 88.1% Nov 2025 Official leaderboard
15 Kimi K3 88% Jul 2026 Official leaderboard
16 GPT-6 Astra 87.9% Sep 2026 Official leaderboard
17 Gemini 4 Argon 87.4% Sep 2026 Official leaderboard
18 MiniMax-M3 87.3% Jun 2026 Official leaderboard
19 Grok 4.5 86.9% Jul 2026 Official leaderboard
20 GPT-5.5 86.9% Apr 2026 Official leaderboard
21 Claude Opus 4.6 86.7% non-thinking Feb 2026 Official leaderboard
22 Grok 4.6 86.5% Aug 2026 Official leaderboard
23 GPT-6.1 Sol 86.5% Sep 2026 Official leaderboard
24 Muse Spark 85.9% Apr 2026 Official leaderboard
25 Claude Opus 4.8 85.8% May 2026 Official leaderboard
26 DeepSeek V4.1 Flash 85.5% Sep 2026 Official leaderboard
27 Inkling 85.4% Jul 2026 Official leaderboard
28 Claude Opus 4.5 85.3% thinking Nov 2025 Official leaderboard
29 MiMo-V2.6 Flash 85.3% Sep 2026 Official leaderboard
30 Claude Haiku 4.5 85.2% thinking Oct 2025 Official leaderboard
31 GPT-5.6 Sol 85.2% Jul 2026 Official leaderboard
32 Qwen 3.8 Max 85% Aug 2026 Official leaderboard
33 Claude Sonnet 4.5 84.5% non-thinking Sep 2025 Official leaderboard
34 Gemini 3.8 Flash 84.5% Sep 2026 Official leaderboard
35 GPT-5.2 84.4% Dec 2025 Official leaderboard
36 GPT-5.6 Luna 84.4% Jul 2026 Official leaderboard
37 Inkling-Small 84.1% Jul 2026 Official leaderboard
38 Gemini 3.7 Flash 83.9% Aug 2026 Official leaderboard
39 Qwen 3.8 27B 83.8% Aug 2026 Official leaderboard
40 MiMo-V2.5-Pro 83.7% Apr 2026 Official leaderboard
41 GPT-6 Luna 83.7% Sep 2026 Official leaderboard
42 GPT-5 83.7% Aug 2025 Official leaderboard
43 Tencent Hy4 preview 83.6% Aug 2026 Official leaderboard
44 GLM-5.2 83.5% Jun 2026 Official leaderboard
45 Gemini 2.5 Flash Jun 2025 83% thinking Jun 2025 Official leaderboard
46 Claude Opus 4.7 83% Apr 2026 Official leaderboard
47 GPT-5.6 Terra 82.9% Jul 2026 Official leaderboard
48 GPT-6 Sol 82% Sep 2026 Official leaderboard
49 Grok 4 Fast 81.6% reasoning Sep 2025 Official leaderboard
50 Ling-3.0-flash 80.9% Aug 2026 Official leaderboard
51 MiniMax-M2.1 80.8% Dec 2025 Official leaderboard
52 GPT-5 mini 80.6% Aug 2025 Official leaderboard
53 Mistral Large 4 80.4% Oct 2026 Official leaderboard
54 DeepSeek V4 Flash 0731 80.4% Jul 2026 Official leaderboard
55 DeepSeek V4 Pro 0813 80.2% Aug 2026 Official leaderboard
56 MiniMax-M2.7 79.9% Mar 2026 Official leaderboard
57 Gemini 3.6 Flash 79.7% Jul 2026 Official leaderboard
58 Qwen3.7-Max 79.4% May 2026 Official leaderboard
59 Grok 4.1 Fast 78.7% reasoning Nov 2025 Official leaderboard
60 Gemini 2.5 Flash Sep 2025 78.5% thinking Sep 2025 Official leaderboard
61 Grok 4 78.2% Jul 2025 Official leaderboard
62 Kimi K2.6 78.2% Apr 2026 Official leaderboard
63 GPT-5.4 77.5% xhigh Mar 2026 Official leaderboard
64 Qwen3-VL-Plus 77.1% Sep 2025 Official leaderboard
65 GPT-5.4 Nano 77.1% Mar 2026 Official leaderboard
66 Qwen 3.6 Plus 77% Mar 2026 Official leaderboard
67 o3 76.7% Apr 2025 Official leaderboard
68 Gemini 3.5 Flash 76.6% May 2026 Official leaderboard
69 Kimi K2.5 76.4% thinking Jan 2026 Official leaderboard
70 Gemini 3.1 Pro 76.1% Feb 2026 Official leaderboard
71 Claude Sonnet 5 76% Jun 2026 Official leaderboard
72 Gemini 2.5 Flash-Lite Sep 2025 75.8% non-thinking Sep 2025 Official leaderboard
73 Ling-3.0-flash-Fin 75.6% Sep 2026 Official leaderboard
74 DeepSeek-V4-Pro 75.1% Apr 2026 Official leaderboard
75 Grok 4.3 74.4% Apr 2026 Official leaderboard
76 Claude Opus 4.1 73.9% thinking Aug 2025 Official leaderboard
77 Gemini 2.5 Pro Jun 2025 73.5% Jun 2025 Official leaderboard
78 GPT-5 nano 72.9% Aug 2025 Official leaderboard
79 Qwen3-Max 72.7% Sep 2025 Official leaderboard
80 Claude Sonnet 4 72.4% non-thinking May 2025 Official leaderboard
81 GLM-5.1 72.3% Apr 2026 Official leaderboard
82 MiMo-V2.5 72.2% Apr 2026 Official leaderboard
83 Gemini 3 Pro 72% Nov 2025 Official leaderboard
84 Gemini 3.5 Flash-Lite 70.9% Jul 2026 Official leaderboard
85 Qwen 3.5 Flash (hosted 35B-A3B) 70.6% Feb 2026 Official leaderboard
86 Gemini 3 Flash 69.9% Dec 2025 Official leaderboard
87 o4-mini 69.1% Apr 2025 Official leaderboard
88 GLM-4.7 68.6% Dec 2025 Official leaderboard
89 Mistral Medium 3.5 67.7% Apr 2026 Official leaderboard
90 Laguna M.1 65.9% Apr 2026 Official leaderboard
91 Gemini 3.1 Flash-Lite 63.9% Mar 2026 Official leaderboard
92 Grok 4.20 63.4% reasoning Feb 2026 Official leaderboard
93 Laguna XS.2 61.4% Apr 2026 Official leaderboard
94 Command A+ 55.7% May 2026 Official leaderboard
95 Mercury 2.5 55.1% Sep 2026 Official leaderboard
96 Llama 4 Maverick 54.2% Apr 2025 Official leaderboard
97 Llama 4 Scout 50.6% Apr 2025 Official leaderboard
98 Nemotron 3.5 Lightning 30B-A3B 4.27% Aug 2026 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 4.6
thinking
86.7%
Claude Opus 4.5
non-thinking thinking
85.3%
Claude Sonnet 4.5
thinking
84.5%
Gemini 2.5 Flash
non-thinking
83%
Grok 4 Fast
non-reasoni… reasoning
81.6%
Grok 4.1 Fast
non-reasoni…
78.7%
Gemini 2.5 Flash
non-thinking
78.5%
Gemini 2.5 Flash-Lite
thinking non-thinking non-thinking
75.8%
Claude Opus 4.1
non-thinking thinking
73.9%
Claude Sonnet 4
thinking non-thinking
72.4%
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 91.4%
Claude Fable 5.1 91.3%
Claude Sonnet 5.5 91.1%
Claude Opus 5 91%
Muse Spark 1.2 90.1%
Claude Haiku 5.5 89.5%
Grok 4.7 89.4%
Muse Wonder Full 89.3%
GLM-5.3-Flash 88.9%
Muse Spark 1.1 88.9%
GLM-5.3 88.8%
Claude Fable 5 88.5%
MiMo-V2.6 Pro 88.3%
GPT-5.1 88.1%
Kimi K3 88%
GPT-6 Astra 87.9%
Gemini 4 Argon 87.4%
MiniMax-M3 87.3%
Grok 4.5 86.9%
GPT-5.5 86.9%
Claude Opus 4.6 non-thinking 86.7%
Grok 4.6 86.5%
GPT-6.1 Sol 86.5%
Claude Opus 4.6 thinking 86.1%
Muse Spark 85.9%
Claude Opus 4.8 85.8%
DeepSeek V4.1 Flash 85.5%
Inkling 85.4%
Claude Opus 4.5 thinking 85.3%
MiMo-V2.6 Flash 85.3%
Claude Haiku 4.5 thinking 85.2%
GPT-5.6 Sol 85.2%
Qwen 3.8 Max 85%
Claude Sonnet 4.5 non-thinking 84.5%
Gemini 3.8 Flash 84.5%
GPT-5.2 84.4%
GPT-5.6 Luna 84.4%
Inkling-Small 84.1%
Claude Sonnet 4.5 thinking 84.1%
Gemini 3.7 Flash 83.9%
Qwen 3.8 27B 83.8%
MiMo-V2.5-Pro 83.7%
GPT-6 Luna 83.7%
GPT-5 83.7%
Tencent Hy4 preview 83.6%
GLM-5.2 83.5%
Claude Opus 4.5 non-thinking 83.3%
Gemini 2.5 Flash Jun 2025 thinking 83%
Claude Opus 4.7 83%
Gemini 2.5 Flash Jun 2025 non-thinking 82.9%
GPT-5.6 Terra 82.9%
GPT-6 Sol 82%
Grok 4 Fast reasoning 81.6%
Ling-3.0-flash 80.9%
MiniMax-M2.1 80.8%
GPT-5 mini 80.6%
Mistral Large 4 80.4%
DeepSeek V4 Flash 0731 80.4%
DeepSeek V4 Pro 0813 80.2%
MiniMax-M2.7 79.9%
Grok 4 Fast non-reasoning 79.7%
Gemini 3.6 Flash 79.7%
Qwen3.7-Max 79.4%
Grok 4.1 Fast reasoning 78.7%
Gemini 2.5 Flash Sep 2025 thinking 78.5%
Grok 4 78.2%
Kimi K2.6 78.2%
Gemini 2.5 Flash Sep 2025 non-thinking 78%
GPT-5.4 xhigh 77.5%
Grok 4.1 Fast non-reasoning 77.5%
Qwen3-VL-Plus 77.1%
GPT-5.4 Nano 77.1%
Qwen 3.6 Plus 77%
o3 76.7%
Gemini 3.5 Flash 76.6%
Kimi K2.5 thinking 76.4%
Gemini 3.1 Pro 76.1%
Claude Sonnet 5 76%
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 75.8%
Ling-3.0-flash-Fin 75.6%
DeepSeek-V4-Pro 75.1%
Grok 4.3 74.4%
Claude Opus 4.1 thinking 73.9%
Gemini 2.5 Pro Jun 2025 73.5%
GPT-5 nano 72.9%
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 72.8%
Qwen3-Max 72.7%
Claude Sonnet 4 non-thinking 72.4%
GLM-5.1 72.3%
MiMo-V2.5 72.2%
Gemini 3 Pro 72%
Claude Opus 4.1 non-thinking 71.8%
Gemini 3.5 Flash-Lite 70.9%
Qwen 3.5 Flash (hosted 35B-A3B) 70.6%
Gemini 3 Flash 69.9%
Claude Sonnet 4 thinking 69.3%
o4-mini 69.1%
GLM-4.7 68.6%
Mistral Medium 3.5 67.7%
Gemini 2.5 Flash-Lite Sep 2025 thinking 66.9%
Laguna M.1 65.9%
Gemini 3.1 Flash-Lite 63.9%
Grok 4.20 reasoning 63.4%
Laguna XS.2 61.4%
Command A+ 55.7%
Mercury 2.5 55.1%
Llama 4 Maverick 54.2%
Llama 4 Scout 50.6%
Nemotron 3.5 Lightning 30B-A3B 4.27%

6 models have held number one since its release in Feb 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Anthropic Meta
Own leaderboard Passed 85% in Feb 2026, by GPT-5.1
Claude Opus 5.5 91.4%

The best score passed 85% in Oct 2025

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 5.5 GPT-5.1 GPT-5 o3 Llama 4 Maverick 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 5.5 Claude Sonnet 4.5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 140 in Law, finance and medicine
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%
  • PRBench In use Model answers 1,100 expert-written law and finance questions. LLM judge grades against weighted expert rubrics of 10-30 criteria each. Muse Spark 1.3 61.6%