KBank-MMLU (Upstage)
Solar Open 2 80.8% leads since Jul 2026 Fading Models answer Korean finance questions in a multiple-choice format. Upstage built the set in house.
The technical report lists it as the finance part of the professional domain group, with no further detail.
- Level
- Expert Specialists with years of training, often over hours of work
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Multiple choice
Solar Open 2 leads with 80.8% among models released by Jul 2026
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
The newest model with a score here came out in Jul 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 16 more
- 1 Solar Open 2 80.8%
- 2 Solar Pro 2 71%
- 3 Solar Open 100B 65.5%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Solar Open 2 | 80.8% | Jul 2026 | Model card | |
| 2 | Solar Pro 2 | 71% | Jul 2025 | Model card | |
| 3 | Solar Open 100B | 65.5% | Dec 2025 | Model card |
2 models have held number one since its release in Jul 2026
Lab claims count from the day the card came out, independent runs from the model’s release.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Labs headline it less often now, last in Jul 2026
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 22 Jul 2026 | Solar Open 2 | 80.8 | KBank-MMLU (in-house) |
| 31 Dec 2025 | Solar Open 100B | 65.5 | KBankMMLU (in-house) |
| 10 Jul 2025 | Solar Pro 2 | 71 | KBankMMLU (finance, private benchmark built with Korean financial institutions) |
2 system cards report it, from 1 lab
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 22 Jul 2026 | Solar Open 2 technical report | 80.8 | KBank-MMLU, in-house (Table 4) | Capabilities |
| 11 Jan 2026 | Solar Open 100B technical report | 65.5 | KBankMMLU (in-house), finance (Table 5) | Capabilities |
More in Law, finance and medicine
- HealthBench Professional In use 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. Claude Sonnet 5.5 69.2%
- MedXpertQA-MM In use Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy.
Qwen 3.8 Max 80.4% - HealthBench In use 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. GPT-5 67.2%
- KBL In use Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. Solar Pro 4 77.5%
- KorMedMCQA In use Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. Solar Pro 4 93.2%
- PRBench In use Model answers 1,100 expert-written law and finance questions. LLM judge grades against weighted expert rubrics of 10-30 criteria each.
Muse Spark 1.3 61.6%