Benchmark Almanac

MedCode

Claude Opus 5 63.6% leads since Jul 2026 Not in model cards

Models read de-identified patient records and output primary and secondary ICD-10-CM diagnosis codes. Answers are checked against codes assigned by professional coders.

Records include discharge summaries with progress and consult notes. Coders used the Codify lookup tool but no AI. 107 models on 2026-10-29, led by Claude Opus 5 at 63.57%.

Made by Vals AI Released Feb 2026 Medicine Tasks kept private Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
Leaderboard
96
Models scored on its own leaderboard

Claude Opus 5 leads with 63.6%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

96 models. Scroll sideways for the rest.

  1. 63.6 Claude Opus 5
  2. 59.1 Gemini 3.1 Pro
  3. 58.8 Gemini 4 Argon
  4. 56.1 Claude Fable 5
  5. 55.9 Gemini 3 Flash
  6. 55.8 Gemini 3.5 Flash
  7. 54.9 Claude Opus 4.7
  8. 53.5 Claude Fable 5.1
  9. 53.4 Gemini 3.7 Flash
  10. 53.2 Claude Opus 4.8
  11. 53.1 Gemini 3.6 Flash
  12. 52.9 Claude Sonnet 5.5
  13. 52.7 GPT-5.1
  14. 52.2 Gemini 3 Pro
  15. 51.5 Claude Haiku 5.5
  16. 51.3 Muse Spark
  17. 50.6 Gemini 2.5 Pro
  18. 49.9 Muse Wonder Full
  19. 49.8 Claude Opus 5.5
  20. 49.8 GPT-5.2
  21. 49.6 GPT-5
  22. 49.5 Grok 4.7
  23. 49.4 Kimi K3
  24. 49.4 Muse Spark 1.2
  25. 49.2 Claude Opus 4.5
  26. 49.1 Claude Opus 4.6
  27. 49.1 GPT-5.5
  28. 48.8 GPT-6.1 Sol
  29. 48.5 GPT-6 Astra
  30. 48.1 Gemini 3.8 Flash
  31. 47.6 Gemini 3.1 Flash-Lite
  32. 47.5 Claude Sonnet 5
  33. 47.3 o3
  34. 47.2 Claude Opus 4.1
  35. 47.1 GPT-6 Sol
  36. 46.3 MiniMax-M3
  37. 45.0 MiMo-V2.6 Pro
  38. 44.7 Grok 4.6
  39. 44.7 GPT-6 Luna
  40. 44.1 Claude Sonnet 4.5
  41. 44.0 GPT-5.6 Sol
  42. 43.5 Gemini 3.5 Flash-Lite
  43. 43.4 GPT-5.6 Terra
  44. 43.3 Grok 4.5
  45. 43.3 Tencent Hy4 preview
  46. 43.0 GPT-5 mini
  47. 42.9 GLM-5.3
  48. 42.5 DeepSeek V4 Pro 0813
  49. 42.4 GPT-5.6 Luna
  50. 41.6 GLM-5.1
  51. 41.4 DeepSeek V4 Flash 0731
  52. 41.3 GPT-5.4
  53. 41.2 Inkling
  54. 41.2 DeepSeek V4.1 Flash
  55. 41.1 MiMo-V2.6 Flash
  56. 41.0 GPT-5.4 Nano
  57. 40.8 GLM-5.2
  58. 40.7 Mistral Large 4
  59. 40.7 Qwen 3.8 Max
  60. 40.5 Gemini 2.5 Flash
  61. 40.5 DeepSeek-V4-Pro
  62. 40.4 Gemini 2.5 Flash
  63. 40.1 Kimi K2.6
  64. 39.3 Kimi K2.5
  65. 38.8 Qwen3.7-Max
  66. 38.6 Nemotron 3 Ultra
  67. 38.1 Grok 4
  68. 38.1 Grok 4.3
  69. 37.9 Inkling-Small
  70. 37.4 Grok 4 Fast
  71. 36.9 Qwen 3.6 Plus
  72. 36.5 Llama 4 Maverick
  73. 35.0 Claude Sonnet 4
  74. 34.4 MiniMax-M2.7
  75. 34.2 Gemini 2.5 Flash-Lite
  76. 34.1 MiniMax-M2.1
  77. 33.8 o4-mini
  78. 33.8 Mistral Medium 3.5
  79. 33.0 Qwen 3.5 Flash (hosted 35B-A3B)
  80. 32.8 GLM-4.7
  81. 32.7 Claude Haiku 4.5
  82. 32.5 MiMo-V2.5-Pro
  83. 32.3 Ling-3.0-flash
  84. 32.2 Grok 4.20
  85. 31.9 MiMo-V2.5
  86. 31.6 Qwen3-VL-Plus
  87. 31.4 Qwen3-Max
  88. 31.3 Mercury 2.5
  89. 30.4 GPT-5 nano
  90. 29.3 Ling-3.0-flash-Fin
  91. 28.7 Qwen 3.8 27B
  92. 28.4 Grok 4.1 Fast
  93. 23.3 Llama 4 Scout
  94. 23.1 Laguna M.1
  95. 21.3 Laguna XS.2
  96. 19.7 Command A+
  1. 1 Claude Opus 5 63.6%
  2. 2 Gemini 3.1 Pro 59.1%
  3. 3 Gemini 4 Argon 58.8%
  4. 4 Claude Fable 5 56.1%
  5. 5 Gemini 3 Flash 55.9%
  6. 6 Gemini 3.5 Flash 55.8%
  7. 7 Claude Opus 4.7 54.9%
  8. 8 Claude Fable 5.1 53.5%
  9. 9 Gemini 3.7 Flash 53.4%
  10. 10 Claude Opus 4.8 53.2%
  11. 11 Gemini 3.6 Flash 53.1%
  12. 12 Claude Sonnet 5.5 52.9%
  13. 13 GPT-5.1 52.7%
  14. 14 Gemini 3 Pro 52.2%
  15. 15 Claude Haiku 5.5 51.5%
  16. 16 Muse Spark 51.3%
  17. 17 Gemini 2.5 Pro Jun 2025 50.6%
  18. 18 Muse Wonder Full 49.9%
  19. 19 Claude Opus 5.5 49.8%
  20. 20 GPT-5.2 49.8%
  21. 21 GPT-5 49.6%
  22. 22 Grok 4.7 49.5%
  23. 23 Kimi K3 49.4%
  24. 24 Muse Spark 1.2 49.4%
  25. 25 Claude Opus 4.5 thinking 49.2%
  26. 26 Claude Opus 4.6 thinking 49.1%
  27. 27 GPT-5.5 49.1%
  28. 28 GPT-6.1 Sol 48.8%
  29. 29 GPT-6 Astra 48.5%
  30. 30 Gemini 3.8 Flash 48.1%
  31. 31 Gemini 3.1 Flash-Lite 47.6%
  32. 32 Claude Sonnet 5 47.5%
  33. 33 o3 47.3%
  34. 34 Claude Opus 4.1 thinking 47.2%
  35. 35 GPT-6 Sol 47.1%
  36. 36 MiniMax-M3 46.3%
  37. 37 MiMo-V2.6 Pro 45%
  38. 38 Grok 4.6 44.7%
  39. 39 GPT-6 Luna 44.7%
  40. 40 Claude Sonnet 4.5 thinking 44.1%
  41. 41 GPT-5.6 Sol 44%
  42. 42 Gemini 3.5 Flash-Lite 43.5%
  43. 43 GPT-5.6 Terra 43.4%
  44. 44 Grok 4.5 43.3%
  45. 45 Tencent Hy4 preview 43.3%
  46. 46 GPT-5 mini 43%
  47. 47 GLM-5.3 42.9%
  48. 48 DeepSeek V4 Pro 0813 42.5%
  49. 49 GPT-5.6 Luna 42.4%
  50. 50 GLM-5.1 41.6%
  51. 51 DeepSeek V4 Flash 0731 41.4%
  52. 52 GPT-5.4 xhigh 41.3%
  53. 53 Inkling 41.2%
  54. 54 DeepSeek V4.1 Flash 41.2%
  55. 55 MiMo-V2.6 Flash 41.1%
  56. 56 GPT-5.4 Nano 41%
  57. 57 GLM-5.2 40.8%
  58. 58 Mistral Large 4 40.7%
  59. 59 Qwen 3.8 Max 40.7%
  60. 60 Gemini 2.5 Flash Sep 2025, non-thinking 40.5%
  61. 61 DeepSeek-V4-Pro 40.5%
  62. 62 Gemini 2.5 Flash Jun 2025, thinking 40.4%
  63. 63 Kimi K2.6 40.1%
  64. 64 Kimi K2.5 thinking 39.3%
  65. 65 Qwen3.7-Max 38.8%
  66. 66 Nemotron 3 Ultra 38.6%
  67. 67 Grok 4 38.1%
  68. 68 Grok 4.3 38.1%
  69. 69 Inkling-Small 37.9%
  70. 70 Grok 4 Fast reasoning 37.4%
  71. 71 Qwen 3.6 Plus 36.9%
  72. 72 Llama 4 Maverick 36.5%
  73. 73 Claude Sonnet 4 thinking 35%
  74. 74 MiniMax-M2.7 34.4%
  75. 75 Gemini 2.5 Flash-Lite Sep 2025, thinking 34.2%
  76. 76 MiniMax-M2.1 34.1%
  77. 77 o4-mini 33.8%
  78. 78 Mistral Medium 3.5 33.8%
  79. 79 Qwen 3.5 Flash (hosted 35B-A3B) 33%
  80. 80 GLM-4.7 32.8%
  81. 81 Claude Haiku 4.5 thinking 32.7%
  82. 82 MiMo-V2.5-Pro 32.5%
  83. 83 Ling-3.0-flash 32.3%
  84. 84 Grok 4.20 reasoning 32.2%
  85. 85 MiMo-V2.5 31.9%
  86. 86 Qwen3-VL-Plus 31.6%
  87. 87 Qwen3-Max 31.4%
  88. 88 Mercury 2.5 31.3%
  89. 89 GPT-5 nano 30.4%
  90. 90 Ling-3.0-flash-Fin 29.3%
  91. 91 Qwen 3.8 27B 28.7%
  92. 92 Grok 4.1 Fast non-reasoning 28.4%
  93. 93 Llama 4 Scout 23.3%
  94. 94 Laguna M.1 23.1%
  95. 95 Laguna XS.2 21.3%
  96. 96 Command A+ 19.7%
#ModelScoreSettingDateSource
1 Claude Opus 5 63.6% Jul 2026 Official leaderboard
2 Gemini 3.1 Pro 59.1% Feb 2026 Official leaderboard
3 Gemini 4 Argon 58.8% Sep 2026 Official leaderboard
4 Claude Fable 5 56.1% Jun 2026 Official leaderboard
5 Gemini 3 Flash 55.9% Dec 2025 Official leaderboard
6 Gemini 3.5 Flash 55.8% May 2026 Official leaderboard
7 Claude Opus 4.7 54.9% Apr 2026 Official leaderboard
8 Claude Fable 5.1 53.5% Sep 2026 Official leaderboard
9 Gemini 3.7 Flash 53.4% Aug 2026 Official leaderboard
10 Claude Opus 4.8 53.2% May 2026 Official leaderboard
11 Gemini 3.6 Flash 53.1% Jul 2026 Official leaderboard
12 Claude Sonnet 5.5 52.9% Sep 2026 Official leaderboard
13 GPT-5.1 52.7% Nov 2025 Official leaderboard
14 Gemini 3 Pro 52.2% Nov 2025 Official leaderboard
15 Claude Haiku 5.5 51.5% Oct 2026 Official leaderboard
16 Muse Spark 51.3% Apr 2026 Official leaderboard
17 Gemini 2.5 Pro Jun 2025 50.6% Jun 2025 Official leaderboard
18 Muse Wonder Full 49.9% Oct 2026 Official leaderboard
19 Claude Opus 5.5 49.8% Sep 2026 Official leaderboard
20 GPT-5.2 49.8% Dec 2025 Official leaderboard
21 GPT-5 49.6% Aug 2025 Official leaderboard
22 Grok 4.7 49.5% Sep 2026 Official leaderboard
23 Kimi K3 49.4% Jul 2026 Official leaderboard
24 Muse Spark 1.2 49.4% Aug 2026 Official leaderboard
25 Claude Opus 4.5 49.2% thinking Nov 2025 Official leaderboard
26 Claude Opus 4.6 49.1% thinking Feb 2026 Official leaderboard
27 GPT-5.5 49.1% Apr 2026 Official leaderboard
28 GPT-6.1 Sol 48.8% Sep 2026 Official leaderboard
29 GPT-6 Astra 48.5% Sep 2026 Official leaderboard
30 Gemini 3.8 Flash 48.1% Sep 2026 Official leaderboard
31 Gemini 3.1 Flash-Lite 47.6% Mar 2026 Official leaderboard
32 Claude Sonnet 5 47.5% Jun 2026 Official leaderboard
33 o3 47.3% Apr 2025 Official leaderboard
34 Claude Opus 4.1 47.2% thinking Aug 2025 Official leaderboard
35 GPT-6 Sol 47.1% Sep 2026 Official leaderboard
36 MiniMax-M3 46.3% Jun 2026 Official leaderboard
37 MiMo-V2.6 Pro 45% Sep 2026 Official leaderboard
38 Grok 4.6 44.7% Aug 2026 Official leaderboard
39 GPT-6 Luna 44.7% Sep 2026 Official leaderboard
40 Claude Sonnet 4.5 44.1% thinking Sep 2025 Official leaderboard
41 GPT-5.6 Sol 44% Jul 2026 Official leaderboard
42 Gemini 3.5 Flash-Lite 43.5% Jul 2026 Official leaderboard
43 GPT-5.6 Terra 43.4% Jul 2026 Official leaderboard
44 Grok 4.5 43.3% Jul 2026 Official leaderboard
45 Tencent Hy4 preview 43.3% Aug 2026 Official leaderboard
46 GPT-5 mini 43% Aug 2025 Official leaderboard
47 GLM-5.3 42.9% Aug 2026 Official leaderboard
48 DeepSeek V4 Pro 0813 42.5% Aug 2026 Official leaderboard
49 GPT-5.6 Luna 42.4% Jul 2026 Official leaderboard
50 GLM-5.1 41.6% Apr 2026 Official leaderboard
51 DeepSeek V4 Flash 0731 41.4% Jul 2026 Official leaderboard
52 GPT-5.4 41.3% xhigh Mar 2026 Official leaderboard
53 Inkling 41.2% Jul 2026 Official leaderboard
54 DeepSeek V4.1 Flash 41.2% Sep 2026 Official leaderboard
55 MiMo-V2.6 Flash 41.1% Sep 2026 Official leaderboard
56 GPT-5.4 Nano 41% Mar 2026 Official leaderboard
57 GLM-5.2 40.8% Jun 2026 Official leaderboard
58 Mistral Large 4 40.7% Oct 2026 Official leaderboard
59 Qwen 3.8 Max 40.7% Aug 2026 Official leaderboard
60 Gemini 2.5 Flash Sep 2025 40.5% non-thinking Sep 2025 Official leaderboard
61 DeepSeek-V4-Pro 40.5% Apr 2026 Official leaderboard
62 Gemini 2.5 Flash Jun 2025 40.4% thinking Jun 2025 Official leaderboard
63 Kimi K2.6 40.1% Apr 2026 Official leaderboard
64 Kimi K2.5 39.3% thinking Jan 2026 Official leaderboard
65 Qwen3.7-Max 38.8% May 2026 Official leaderboard
66 Nemotron 3 Ultra 38.6% Jun 2026 Official leaderboard
67 Grok 4 38.1% Jul 2025 Official leaderboard
68 Grok 4.3 38.1% Apr 2026 Official leaderboard
69 Inkling-Small 37.9% Jul 2026 Official leaderboard
70 Grok 4 Fast 37.4% reasoning Sep 2025 Official leaderboard
71 Qwen 3.6 Plus 36.9% Mar 2026 Official leaderboard
72 Llama 4 Maverick 36.5% Apr 2025 Official leaderboard
73 Claude Sonnet 4 35% thinking May 2025 Official leaderboard
74 MiniMax-M2.7 34.4% Mar 2026 Official leaderboard
75 Gemini 2.5 Flash-Lite Sep 2025 34.2% thinking Sep 2025 Official leaderboard
76 MiniMax-M2.1 34.1% Dec 2025 Official leaderboard
77 o4-mini 33.8% Apr 2025 Official leaderboard
78 Mistral Medium 3.5 33.8% Apr 2026 Official leaderboard
79 Qwen 3.5 Flash (hosted 35B-A3B) 33% Feb 2026 Official leaderboard
80 GLM-4.7 32.8% Dec 2025 Official leaderboard
81 Claude Haiku 4.5 32.7% thinking Oct 2025 Official leaderboard
82 MiMo-V2.5-Pro 32.5% Apr 2026 Official leaderboard
83 Ling-3.0-flash 32.3% Aug 2026 Official leaderboard
84 Grok 4.20 32.2% reasoning Feb 2026 Official leaderboard
85 MiMo-V2.5 31.9% Apr 2026 Official leaderboard
86 Qwen3-VL-Plus 31.6% Sep 2025 Official leaderboard
87 Qwen3-Max 31.4% Sep 2025 Official leaderboard
88 Mercury 2.5 31.3% Sep 2026 Official leaderboard
89 GPT-5 nano 30.4% Aug 2025 Official leaderboard
90 Ling-3.0-flash-Fin 29.3% Sep 2026 Official leaderboard
91 Qwen 3.8 27B 28.7% Aug 2026 Official leaderboard
92 Grok 4.1 Fast 28.4% non-reasoning Nov 2025 Official leaderboard
93 Llama 4 Scout 23.3% Apr 2025 Official leaderboard
94 Laguna M.1 23.1% Apr 2026 Official leaderboard
95 Laguna XS.2 21.3% Apr 2026 Official leaderboard
96 Command A+ 19.7% May 2026 Official leaderboard

Scores at each effort level

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 4.5
non-thinking thinking
49.2%
Claude Opus 4.6
non-thinking
49.1%
Claude Opus 4.1
non-thinking thinking
47.2%
Claude Sonnet 4.5
non-thinking thinking
44.1%
Gemini 2.5 Flash
thinking
40.5%
Gemini 2.5 Flash
non-thinking thinking
40.4%
Grok 4 Fast
non-reasoni… reasoning
37.4%
Claude Sonnet 4
non-thinking
35%
Gemini 2.5 Flash-Lite
non-thinking thinking
34.2%
Grok 4.1 Fast
reasoning
28.4%
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5 63.6%
Gemini 3.1 Pro 59.1%
Gemini 4 Argon 58.8%
Claude Fable 5 56.1%
Gemini 3 Flash 55.9%
Gemini 3.5 Flash 55.8%
Claude Opus 4.7 54.9%
Claude Fable 5.1 53.5%
Gemini 3.7 Flash 53.4%
Claude Opus 4.8 53.2%
Gemini 3.6 Flash 53.1%
Claude Sonnet 5.5 52.9%
GPT-5.1 52.7%
Gemini 3 Pro 52.2%
Claude Haiku 5.5 51.5%
Muse Spark 51.3%
Gemini 2.5 Pro Jun 2025 50.6%
Muse Wonder Full 49.9%
Claude Opus 5.5 49.8%
GPT-5.2 49.8%
GPT-5 49.6%
Grok 4.7 49.5%
Kimi K3 49.4%
Muse Spark 1.2 49.4%
Claude Opus 4.5 thinking 49.2%
Claude Opus 4.6 thinking 49.1%
GPT-5.5 49.1%
GPT-6.1 Sol 48.8%
GPT-6 Astra 48.5%
Claude Opus 4.6 non-thinking 48.2%
Gemini 3.8 Flash 48.1%
Gemini 3.1 Flash-Lite 47.6%
Claude Sonnet 5 47.5%
o3 47.3%
Claude Opus 4.1 thinking 47.2%
GPT-6 Sol 47.1%
MiniMax-M3 46.3%
Claude Opus 4.5 non-thinking 45.2%
MiMo-V2.6 Pro 45%
Grok 4.6 44.7%
GPT-6 Luna 44.7%
Claude Sonnet 4.5 thinking 44.1%
GPT-5.6 Sol 44%
Gemini 3.5 Flash-Lite 43.5%
GPT-5.6 Terra 43.4%
Grok 4.5 43.3%
Tencent Hy4 preview 43.3%
GPT-5 mini 43%
GLM-5.3 42.9%
DeepSeek V4 Pro 0813 42.5%
GPT-5.6 Luna 42.4%
GLM-5.1 41.6%
DeepSeek V4 Flash 0731 41.4%
Claude Opus 4.1 non-thinking 41.4%
GPT-5.4 xhigh 41.3%
Inkling 41.2%
DeepSeek V4.1 Flash 41.2%
MiMo-V2.6 Flash 41.1%
GPT-5.4 Nano 41%
GLM-5.2 40.8%
Mistral Large 4 40.7%
Qwen 3.8 Max 40.7%
Claude Sonnet 4.5 non-thinking 40.6%
Gemini 2.5 Flash Sep 2025 non-thinking 40.5%
DeepSeek-V4-Pro 40.5%
Gemini 2.5 Flash Jun 2025 thinking 40.4%
Gemini 2.5 Flash Sep 2025 thinking 40.3%
Kimi K2.6 40.1%
Kimi K2.5 thinking 39.3%
Qwen3.7-Max 38.8%
Nemotron 3 Ultra 38.6%
Gemini 2.5 Flash Jun 2025 non-thinking 38.4%
Grok 4 38.1%
Grok 4.3 38.1%
Inkling-Small 37.9%
Grok 4 Fast reasoning 37.4%
Qwen 3.6 Plus 36.9%
Llama 4 Maverick 36.5%
Claude Sonnet 4 thinking 35%
MiniMax-M2.7 34.4%
Gemini 2.5 Flash-Lite Sep 2025 thinking 34.2%
MiniMax-M2.1 34.1%
Claude Sonnet 4 non-thinking 33.9%
o4-mini 33.8%
Mistral Medium 3.5 33.8%
Qwen 3.5 Flash (hosted 35B-A3B) 33%
GLM-4.7 32.8%
Claude Haiku 4.5 thinking 32.7%
MiMo-V2.5-Pro 32.5%
Ling-3.0-flash 32.3%
Grok 4.20 reasoning 32.2%
MiMo-V2.5 31.9%
Qwen3-VL-Plus 31.6%
Qwen3-Max 31.4%
Mercury 2.5 31.3%
GPT-5 nano 30.4%
Grok 4 Fast non-reasoning 30%
Ling-3.0-flash-Fin 29.3%
Qwen 3.8 27B 28.7%
Grok 4.1 Fast non-reasoning 28.4%
Grok 4.1 Fast reasoning 28.1%
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 27.1%
Gemini 2.5 Flash-Lite Sep 2025 non-thinking 27.1%
Llama 4 Scout 23.3%
Laguna M.1 23.1%
Laguna XS.2 21.3%
Command A+ 19.7%

3 models have held number one since its release in Feb 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Google DeepMind Anthropic
Own leaderboard 3 leaders
Claude Opus 5 63.6%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 5 Gemini 3.1 Pro GPT-5.1 Gemini 2.5 Pro Llama 4 Maverick 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Opus 5 Gemini 3 Flash

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Law, finance and medicine

All 140 in Law, finance and medicine
  • HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
  • MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. Qwen 3.8 Max 80.4%
  • HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
  • KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
  • KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%
  • PRBench In use Model answers 1,100 expert-written law and finance questions. LLM judge grades against weighted expert rubrics of 10-30 criteria each. Muse Spark 1.3 61.6%