Law, finance and medicine
We track 140 benchmarks in law, finance and medicine, or 154 if you count each version. They run from everyday level to research frontier level.
Anthropic held the top law, finance and medicine score 36% of the contested time
That's about 16 of the 46 months since January 2023, averaged over the benchmarks below. The easiest benchmarks are at the top. Versions of one test share a row, with a dark tick at each new version. We stop the clock, and fade the rest of the row, when the best score passes 85%, or at the newest score if 45 days go by without another.
Anthropic Google DeepMind Meta OpenAI xAI Other labs Qwen Moonshot AI
Everyday Most adults, without training
Expert Specialists with years of training, often over hours of work
Share of contested time at number one
Anthropic 36% OpenAI 23% 
Google DeepMind 19% xAI 9% 
Meta 6% 
Qwen 4% Other labs 2% Moonshot AI 2%
Google DeepMind 19% xAI 9% Meta 6% Qwen 4% Other labs 2% Moonshot AI 2% Scores come from each benchmark’s own leaderboard, from Epoch AI’s runs and from labs’ reports on their own models. We date Epoch runs by the model’s release and lab reports by when they came out. Labs pick their own settings, so a lab’s claim can beat an independent run of the same model. Turn off lab claims to see independent runs only.
Anthropic Google DeepMind Meta OpenAI xAI Other labs Qwen Moonshot AI
Everyday Most adults, without training
Expert Specialists with years of training, often over hours of work
Share of contested time at number one, independent runs only
Anthropic 34% OpenAI 23% Google DeepMind 18% xAI 9% Meta 8% Qwen 4% Other labs 2% Moonshot AI 2%
Google DeepMind 18% xAI 9% Meta 8% Qwen 4% Other labs 2% Moonshot AI 2% Only each benchmark’s own leaderboard and Epoch AI’s runs, dated by the model’s release. We leave out benchmarks that only labs report.
From everyday to research frontier level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
University
Undergraduates and entry-level professionals
Expert
Specialists with years of training, often over hours of work
- LegalBench
- TaxEval v2
- CorpFin v2
- MedScribe
- MedCode
- Harvey LAB (Legal Agent Benchmark)
- Vals Finance Agent v2
- Legal Research Bench
- Tax Agent Bench
- Excel Modeling Benchmark
- Vals Finance Agent
- CaseLaw v2
- PolicyBench
- Public Benefits Bench
- PRBench
- Vals Finance Agent v1
- HealthBench
- HealthBench Professional
- Excel Modeling Benchmark v1.1
- Vals Finance Agent v2.1
- Harvey LAB v1.1 (HLAB-Vals-1.1)
- MedXpertQA-MM
- LEXam
- HealthBench Hard
- prinzbench
- TaxCalcBench v2
- FailureSensorIQ
- Immigration Bench
- FrontierFinance
- RCM-Bench-1
- HealthBench Consensus
- Legora BAR
- TaxCalcBench
- PhysicianBench
- FinSearchComp
- MedHELM
- RealmPathology
- StockBench
- PharmaBench (micro1)
- RadLE 2.0
- Quant-Arena
- Investment Banking Modeling Tasks (OpenAI)
- KBL
- KorMedMCQA
- LegalReasonBench
- micro1 Medical Index
- Ivo-micro1 ContractBench
- BankerToolBench
- KBank-MMLU (Upstage)
- MedQA
- BigFinanceBench
- MBE (Multistate Bar Examination)
- MedXpertQA
- RedlineBench
- Real-World Finance (Anthropic)
- Finance Modeling
- Finance Modeling Pro
- Finch (FinWorkBench)
- FinStepBench
- HealthBench Consensus Global Health
- HealthBench Consensus Urgent
- Hy-FinAgentBench
- Hy-FinModelBench
- Hy-FinmodelBench v2
- PLawBench
- Real-World Finance v2 (Anthropic)
- ScanBench
- Advanced Sommelier (theory knowledge)
- Certified Sommelier (theory knowledge)
- Finance Bench (Moonshot)
- High-risk health scenarios eval
- Introductory Sommelier (theory knowledge)
- MedCalc-Bench Verified
- Medical Knowledge Self-Assessment Program (MKSAP)
- MedMCQA
- MedQA Mainland China
- MedQA Taiwan
- MedQA USMLE (5 options)
- MMLU Clinical Knowledge
- MMLU Medical Genetics
- MMLU Professional Medicine
- Uniform Bar Exam (MBE+MEE+MPT)
- USMLE Step 1 (sample questions)
- USMLE Step 2 (sample questions)
- USMLE Step 3 (sample questions)
- AgentClinic
- Alpha Arena
- AssetOpsBench
- ATLAS-Finance
- ATLAS-Finance v1.1.0
- BankerToolBench v1.1
- Bedside Bench
- Chest ImaGenome
- ChestX-ray14
- CheXpert
- CHI-Bench
- CliniCARE-Bench
- CT-RATE
- DAYJOB: Accounting
- DAYJOB: Healthcare
- DAYJOB: Law
- DAYJOB: Tax
- EHRNoteQA
- EHRQA (Google)
- EyePACS (diabetic retinopathy grading)
- FailSafeQA
- Finance Reasoning (Snorkel)
- GMAI-MMBench
- Health dynamic conversation eval
- Health language and locale eval
- Health static understanding eval
- Hebbia Financial Services Benchmark
- IndustryBench
- Investor's Last Exam
- LAB: Contracts
- LAB: Diligence
- LAB: US Case Law
- LiveMACE
- MBABench
- MBABench 2.0
- MedAgentBench
- MedXpertQA MM Extended
- MIMIC-CXR (classification)
- MIMIC-CXR report generation
- MindEval
- MindEval2
- MLCR-AA
- MS-CXR-T
- PathMCQA (Google)
- PatientAgentBench
- Radiology's Last Exam (RadLE)
- SheetBench
- SnorkelFinance 2.0
- SnorkelLegal
- SnorkelManufacturing
- SnorkelUnderwrite
- SnorkelUnderwrite 2.0
- US-DermMCQA (Google)
- VQA-RAD
Research frontier
Researchers, on open problems or tasks that take days
By kind
Law
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| LegalBench Models do 162 legal reasoning tasks written by legal professionals, covering six types of legal reasoning such as issue spotting and rule application. | Expert | Checked automatically | 88.6% | Not in cards |
| Harvey LAB (Legal Agent Benchmark) Agent works through legal matter files across 24+ practice areas and produces work product. Expert rubric, task passes only if every criterion passes. | Expert | Graded by a model | 25.4% | In use |
| Legal Research Bench Agents answer US federal and state legal research questions with case-law and web tools. Expert rubrics, all-pass and weighted scores by LLM judge. | Expert | Graded by a model | 55.8% | In use |
| CaseLaw v2 Models answer private questions over recent Canadian court decisions, such as finding key cases and answering across several documents. | Expert | Graded by a model | 79.3% | Not in cards |
| PRBench Model answers 1,100 expert-written law and finance questions. LLM judge grades against weighted expert rubrics of 10-30 criteria each. | Expert | Graded by a model | 61.6% | In use |
| Harvey LAB v1.1 (HLAB-Vals-1.1) Agents produce legal work product from matter files with file tools and docx, pptx and xlsx skills. A task passes only if every expert rubric criterion passes. | Expert | Graded by a model | 27.1% | Not in cards |
| LEXam Models answer 7,537 questions from 340 law exams at Swiss, EU and international law courses, in English and German. Open answers are judged against reference answers, and multiple-choice items are exact match. | Expert | Graded by a model | 70.2% | Not in cards |
| prinzbench Models research fact patterns under U.S. statutes and regulations and hunt for obscure public information online. 33 private questions run three times give a score out of 99. | Expert | Graded by people | 91.0 | Not in cards |
| Immigration Bench Models answer expert-written US immigration law questions from seven personas, such as F-1 students, employers and DACA recipients. An AI grader checks each answer against an expert rubric. | Expert | Graded by a model | 75.4% | Not in cards |
| Legora BAR Models work private legal cases end to end inside Legora's agent harness, with the matter's documents and templates. An LLM judge scores outputs against rubrics written by lawyers. | Expert | Graded by a model | 106.0 | Not in cards |
| KBL Models answer Korean legal knowledge and reasoning tasks and Korean bar exam questions. It has 7 knowledge tasks, 4 reasoning tasks and 2,510 bar exam items. | Expert | Checked automatically |  77.5% | In use |
| LegalReasonBench Models work long-horizon legal matters built by practicing lawyers, writing memoranda and court filings across dependent assignments, in closed-record and open-research settings. | Expert | Graded by a model | 50.8% | Not in cards |
| Ivo-micro1 ContractBench Models review contracts and return redlines across 350 attorney-written tasks in 35 scenarios and five agreement types. Tasks cover first review and replies to a counterparty. | Expert | Graded by a model | 58.3% | Not in cards |
| MBE (Multistate Bar Examination) Models answer the 200 multiple choice questions of an official Multistate Bar Examination practice exam. Scored by accuracy. | Expert | Checked automatically | 85% | System cards only |
| RedlineBench Agent acts as counsel at one turn of a SaaS contract negotiation and returns tracked-change redlines. Three LLM judges grade attorney-written rubrics. | Expert | Graded by a model | 50.5% | Not in cards |
| PLawBench Model handles 850 realistic legal consultation, case-analysis and document-drafting tasks. LLM judge scores against about 12,500 expert rubric items. | Expert | Graded by a model | 73.2% | In use |
| Uniform Bar Exam (MBE+MEE+MPT) Models take a simulated Uniform Bar Exam with multiple-choice, essay and performance test parts. Scored on the 400-point scale. | Expert | Graded by people | 298.0 | System cards only |
| DAYJOB: Law Agents do legal assignments from a short request and a workspace of files. An agent judge applies an expert rubric, and a pass needs 95% of criteria. | Expert | Graded by a model | Not in cards | |
| LAB: Contracts 500 in-house contracting tasks (drafting, redlines, issues lists, escalation memos) across 10 contract categories. All-pass expert-rubric grading. | Expert | Graded by a model | Not in cards | |
| LAB: Diligence Agent researches a synthetic M&A virtual data room (thousands of documents, up to ~80M tokens) and drafts a diligence memo scored against planted-issue rubric criteria. | Expert | Graded by a model | Not in cards | |
| LAB: US Case Law Agents draft a litigation brief for the same party and motion as a real filed federal brief, from the case record. Model judges compare briefs pairwise and wins become Elo ratings. | Expert | Graded by a model | Not in cards | |
| LEXam-hard Agents with legal research tools answer the 518 LEXam open questions that strong open models score lowest on. Questions answerable from memory or unanswerable from their text are removed. | Research frontier | Graded by a model | Not in cards | |
| SnorkelLegal Agents work 200 tasks inside a simulated mid-market law firm. Each request ties to a matter, a procedural stage and a role with set authority. | Expert | Graded by a model | Not in cards |
Finance
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| TaxEval v2 Models answer expert-written tax questions. A judge grades answer correctness and stepwise reasoning against expert solutions. | Expert | Graded by a model | 80.4% | Not in cards |
| CorpFin v2 Models answer expert-written questions about credit agreements that often exceed 200 pages. An LLM judge grades each answer for accuracy. | Expert | Graded by a model | 73.2% | In use |
| Vals Finance Agent v2 927 expert-reviewed research questions on public-company filings. Partial credit only after required checks pass. Private test split. | Expert | Graded by a model | 65.4% | In use |
| MortgageTax Vision models read mortgage tax certificates given as images. They extract the year, parcel number and county, and compute the annualized amount due. | Everyday | Checked automatically | 72.1% | Not in cards |
| Tax Agent Bench Agents research 391 expert-written US corporate tax questions with tools and submit one cited written answer. A judge grades it against a rubric and checks that citations are real. | Expert | Graded by a model | 51.8% | Not in cards |
| Excel Modeling Benchmark Agents build investment banking and private equity financial models in Excel from a written brief and source files. Rubrics give partial credit for tabs, structure, formulas and formatting. | Expert | Graded by a model | 76.7% | Not in cards |
| Vals Finance Agent Agents answer 537 financial analyst questions on public company SEC filings with search and filing tools. A three-vote GPT-5.2 judge grades each final answer. | Expert | Graded by a model | 64.4% | Fading |
| PolicyBench No-tools estimate of US taxes and benefits for 100 households across 18 programs. Headline metric is answers within $1 of PolicyEngine-US. | Expert | Checked automatically | 95.2% | Not in cards |
| Vals Finance Agent v1 Agents answer 537 financial analyst questions on public company SEC filings using search and filing tools. An LLM judge grades each final answer. | Expert | Graded by a model | 60.6% | Not in cards |
| Excel Modeling Benchmark v1.1 Agents build the same 103 Excel financial modeling tasks as EMB v1.0, with the Scratch rubric judge changed to GPT-6 Luna. | Expert | Graded by a model | 76.7% | Not in cards |
| Vals Finance Agent v2.1 Agents answer analyst-grade questions on public company filings, scored with dealbreaker-gated partial credit. A three-model jury grades answers and fails hedged ones. | Expert | Graded by a model | 67.8% | Not in cards |
| TaxCalcBench v2 Models read W-2, 1099 and other PDFs and compute full federal and state returns line by line. A return counts only when every evaluated line is exact. | Expert | Checked automatically | 77.5% | Not in cards |
| FrontierFinance Agents answer 220 investment research queries from finance experts across six use cases of the investor workflow. Three judge models check each answer against 11,543 source-attributed rubrics. | Expert | Graded by a model | 55.9% | In use |
| TaxCalcBench Models compute complete US federal tax returns line by line from taxpayer data, with no tax engine. A return counts only when every evaluated line is exact. | Expert | Checked automatically | 62.8% | Not in cards |
| FinSearchComp Agents search the web to fetch live financial data, look up history, and run multi-step investigations, across 635 expert-written questions. Scored by answer accuracy. | Expert | Graded by a model |  81.7% | Fading |
| StockBench Agents get daily prices, fundamentals and news over months and decide to buy, sell or hold. Scoring uses cumulative return, maximum drawdown and related metrics. | Expert | Settled by events | 2.50% | Not in cards |
| Quant-Arena Models act as quant researchers on 2022 to 2023 China A-share data with bounded feedback, then commit one strategy. A cash-account evaluator with trading costs scores it once on a sealed 2024 window. | Expert | Checked automatically | 51.3 | Not in cards |
| Investment Banking Modeling Tasks (OpenAI) Models build first- to third-year investment banking analyst models, such as three-statement and leveraged buyout models, with formatting and citations. OpenAI grades the outputs internally. | Expert | Graded by a model | 88.6% | Fading |
| BankerToolBench 100 end-to-end junior-banker tasks producing Excel, PowerPoint and Word deliverables. Agentic grader scores ~150 weighted expert rubric criteria. | Expert | Graded by a model |  78.6% | In use |
| KBank-MMLU (Upstage) Models answer Korean finance questions in a multiple-choice format. Upstage built the set in house. | Expert | Checked automatically | 80.8% | Fading |
| BigFinanceBench Agents answer workflow-grounded financial research questions with web search, SEC EDGAR, page fetch and Python. A judge grades each answer against an expert rubric and reference. | Expert | Graded by a model | 51% | In use |
| Real-World Finance (Anthropic) Agents complete about 50 analyst tasks in banking, private equity, hedge funds and corporate finance, producing spreadsheets, slides and documents. Rubric graded. | Expert | Graded by a model | 70% | System cards only |
| Finance Modeling Agents complete expert-written financial modeling problems with Excel tools, from research to analysis. Expert-designed rubrics score each problem, averaged over three runs. | Expert | Graded by a model |  21.6% | Fading |
| Finance Modeling Pro Agents complete financial modeling tasks in Excel. Expert rubrics score each task over three trials. | Expert | Graded by a model | 57% | System cards only |
| Finch (FinWorkBench) Agent completes 172 enterprise finance and accounting spreadsheet workflows (384 tasks) from real data. Human plus calibrated LLM-judge grading. | Expert | Graded by people |  73.3% | In use |
| FinStepBench Models do three kinds of finance work: retrieving and checking live financial facts, building company valuations, and writing full research reports. Graders check accuracy, sourcing and reproducible calculations. | Expert | Graded by a model |  63.6% | In use |
| Hy-FinAgentBench Agents complete Tencent in-house finance analyst tasks. Task and scoring details are not published. | Expert | Checked automatically | 79.7% | In use |
| Hy-FinModelBench Agents build financial models for Tencent in-house finance tasks. Task and scoring details are not published. | Expert | Checked automatically | 69% | In use |
| Hy-FinmodelBench v2 Agents build financial models for a second version of Tencent in-house finance tasks. Task and scoring details are not published. | Expert | Checked automatically | 57% | In use |
| Real-World Finance v2 (Anthropic) Agents complete 294 long-horizon finance tasks such as building financial models. A model grader compares deliverables pairwise and ratings are fit as Elo. | Expert | Graded by a model | 1,374 | System cards only |
| Finance Bench (Moonshot) Agents complete realistic financial workflows end to end, from source materials to deliverables that reviewers can check. | Expert | Graded by a model | 62.6% | System cards only |
| FinanceReasoning Models solve financial numerical reasoning problems that need one or more finance formulas. Answers are numbers checked against reference Python solutions. | University | Checked automatically | 87.4% | System cards only |
| AGORA (Agentic Global Open Research Arena) Autonomous AI systems get one secret open finance question, 72 hours and a fixed budget to write a paper. Five finance scholars judge it. | Research frontier | Graded by people | Not in cards | |
| Alpha Arena Frontier models trade live markets with real money over a season, and the leaderboard ranks them by returns. | Expert | Settled by events | Not in cards | |
| ATLAS-Finance Agents do 100 investment banking tasks inside 13 simulated firms. They gather facts from email, chat, calendar and files over MCP, then build an Excel model graded against expert rubrics. | Expert | Graded by a model | Not in cards | |
| ATLAS-Finance v1.1.0 Agents do the same 100 ATLAS-Finance banking tasks, graded with revised rubrics. Weights use a 1 to 10 scale and tolerances and gates sit at the section level. | Expert | Graded by a model | Not in cards | |
| BankerToolBench v1.1 Agents complete 100 junior investment banker tasks and produce Excel, PowerPoint and Word deliverables. A task passes only if every critical rubric criterion passes. | Expert | Graded by a model | Not in cards | |
| ClaimBench Models label whether a 10-K excerpt supports, contradicts or says nothing about a claim, across 1,960 claims. Small decision models are scored on accuracy, cost and latency. | University | Checked automatically | Not in cards | |
| DAYJOB: Accounting Agents do accounting assignments from a short request and a workspace of files. An agent judge applies an expert rubric, and a pass needs 95% of criteria. | Expert | Graded by a model | Not in cards | |
| DAYJOB: Tax Agents take tax assignments from client request to finished return, workpaper or advice, using messy records. A pass needs 95% of the expert rubric criteria. | Expert | Graded by a model | Not in cards | |
| FailSafeQA Models answer finance questions over long documents while the query or the document is degraded, irrelevant or missing. An LLM judge scores robustness, context grounding and compliance. | Expert | Graded by a model | Not in cards | |
| Finance Reasoning (Snorkel) Agents answer expert-written financial reasoning questions grounded in 10-K tables loaded into a database, using SQL tools over many steps. A judge scores correctness and completeness. | Expert | Graded by a model | Not in cards | |
| FinQA (Snorkel OpenEnv environment) Agent answers 290 multi-step questions on SEC 10-K data via MCP tools and constrained SQL. Binary reward with 1% numeric tolerance. | University | Checked automatically | Not in cards | |
| Hebbia Financial Services Benchmark Agents in Hebbia's production harness answer questions over deal and research documents, build Excel models and make client decks. A run passes only if it meets every rubric criterion. | Expert | Graded by a model | Not in cards | |
| Investor's Last Exam Financial agents make equity investment judgments from a frozen set of decision-time evidence. Predictions are scored against later outcomes and professional criteria. | Expert | Graded by people | Not in cards | |
| LiveMACE Agents trade in live financial markets, and the benchmark scores the process capabilities behind their decisions, not only returns. | Expert | Checked automatically | Not in cards | |
| MBABench Agents build complete financial models in Excel from a case prompt and documents. An LLM judge grades accuracy, formulas and formatting on a 0 to 100 scale. | Expert | Graded by a model | Not in cards | |
| MBABench 2.0 Agents build spreadsheet financial models for 101 tasks written by finance experts. A task passes only if every number matches the reference and the workbook meets a quality bar. | Expert | Checked automatically | Not in cards | |
| SheetBench Agents solve finance and accounting spreadsheet tasks through a browser in a fresh Google Sheet. The score is the share of expected answer cells that match exactly. | Expert | Checked automatically | Not in cards | |
| SnorkelFinance ReAct agents answer expert-verified financial questions built from 10-K filings, using SQL, code and other MCP tools without seeing the documents. A judge scores full traces. | University | Graded by a model | Not in cards | |
| SnorkelFinance 2.0 Agents do 200 investment banking and private equity workflow tasks in a simulated firm with incomplete requests, scattered evidence, compliance rules and required state changes. | Expert | Graded by a model | Not in cards | |
| SnorkelUnderwrite Agents help simulated small business insurance underwriters across six task types in multi-turn chats, using tools and asking the underwriter for missing facts. | Expert | Graded by a model | Not in cards | |
| SnorkelUnderwrite 2.0 Agents do 200 commercial property and casualty underwriting tasks in a simulated insurer with partial information, role permissions and stateful workflows. | Expert | Graded by a model | Not in cards |
Medicine
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| MedScribe Models write SOAP notes from clinical visit transcripts. Notes are graded against expert gold-standard rubrics for required content and section structure. | Expert | Graded by a model | 91.4% | Not in cards |
| MedCode Models read de-identified patient records and output primary and secondary ICD-10-CM diagnosis codes. Answers are checked against codes assigned by professional coders. | Expert | Checked automatically | 63.6% | Not in cards |
| HealthBench 5,000 realistic health conversations graded against physician-written rubrics. A length-adjusted variant penalizes verbosity. | Expert | Graded by a model | 67.2% | In use |
| HealthBench Professional 525 physician-authored tasks from real clinician chats (care consult, documentation, research) with physician-adjudicated rubrics. | Expert | Graded by a model | 69.2% | In use |
| MedXpertQA-MM Models answer expert-level medical exam questions that combine images with clinical records and test results. Scoring is accuracy. | Expert | Checked automatically | 80.4% | In use |
| HealthBench Hard Models answer the 1,000 hardest HealthBench conversations, graded against physician-written rubrics. | Expert | Graded by a model | 46.2% | Fading |
| RCM-Bench-1 Agents handle healthcare revenue cycle work, such as claim submission, billing and eligibility checks, with tools. Checks cover tool calls, arguments, replies and final system state. | Expert | Checked automatically | 92.7% | Not in cards |
| HealthBench Consensus Models answer HealthBench conversations graded only on criteria that several physicians agreed on. | Expert | Graded by a model | 96.6% | Dropped |
| PhysicianBench 100 physician tasks performed by an agent inside an EHR environment. Scored by pass rate per attempt. | Expert | Checked automatically | 50% | System cards only |
| MedHELM Models run a suite of medical tasks chosen by clinicians across five categories, from clinical decision support and note writing to patient messages and administration. | Expert | Graded by a model | 70.3 | Not in cards |
| RealmPathology Models interpret real anatomic pathology reports and must not claim a diagnosis, stage or treatment the report does not support. Rubrics have 772 positive and 299 penalty criteria. | Expert | Graded by a model | 78% | Not in cards |
| PharmaBench (micro1) Agents write clinical trial documentation inside 8 synthetic drug programs, across 120 tasks written by biostatisticians, medical writers, safety physicians and regulatory staff. | Expert | Graded by a model | 27.5% | Not in cards |
| RadLE 2.0 Models diagnose single-image radiology cases with a confidence score from 0 to 4, or say they do not know. A confidence-weighted index rewards right answers and penalizes confident errors. | Expert | Checked automatically | 758.0 | Not in cards |
| KorMedMCQA Models answer 7,469 multiple-choice questions from Korean licensing exams for doctors, nurses, pharmacists and dentists, held from 2012 to 2024. | Expert | Checked automatically | 93.2% | In use |
| micro1 Medical Index Models are scored on the equal-weight average of micro1 medical benchmarks, currently RealmPathology, PharmaBench and WellnessBench. | Expert | Graded by a model | 71% | Not in cards |
| PMC-VQA Models answer multiple-choice questions about medical images drawn from PubMed Central figures. Scored by accuracy. | University | Checked automatically | 66.2% | Fading |
| SLAKE Models answer English and Chinese questions about radiology images of the head, neck, chest, abdomen, and pelvis. Scoring is accuracy. | University | Checked automatically | 90.8% | Fading |
| PubMedQA Models answer biomedical research questions with yes, no, or maybe, given a PubMed abstract without its conclusion. Scored by accuracy. | University | Checked automatically | 78.3% | Dropped |
| MedQA Models answer multiple-choice questions from medical licensing exams, usually the 12,723-question English USMLE set. Scoring is accuracy. | Expert | Checked automatically | 89.4% | Dropped |
| MedXpertQA Models answer 4,460 expert-level medical exam questions across 17 specialties, in a text subset and an image subset. Scoring is multiple-choice accuracy. | Expert | Checked automatically | 52.6% | Fading |
| HealthBench Consensus Global Health Models answer health conversations that need adjustment for global context such as local care access. Scored as the failure rate. | Expert | Graded by a model | 0% | System cards only |
| HealthBench Consensus Urgent Models answer ambiguous high-stakes health conversations validated by two or more physicians. Scored as the error rate in informing the user. | Expert | Graded by a model | 0.40% | System cards only |
| ScanBench Models act as doctors in 303 simulated clinical cases from 12 departments. They take a history, order lab tests and give a final diagnosis, scored per stage. | Expert | Graded by a model |  73.8% | Fading |
| High-risk health scenarios eval Adversarial health emergencies test whether models recognize urgency and red flags and direct users to help. Reports pass rate. | Expert | Graded by a model | 98.9% | System cards only |
| MedCalc-Bench Verified Models extract values from patient notes and compute results for 55 medical calculators. Verified release of MedCalc-Bench with corrected labels. | Expert | Checked automatically | 86.2% | System cards only |
| Medical Knowledge Self-Assessment Program (MKSAP) Models answer internal medicine board review questions from MKSAP. Scoring is accuracy. | Expert | Checked automatically | 75% | System cards only |
| MedMCQA Models answer multiple-choice questions from Indian medical entrance exams, AIIMS and NEET PG. Scoring is accuracy. | Expert | Checked automatically | 77% | System cards only |
| MedQA Mainland China Models answer mainland China medical licensing exam questions in simplified Chinese. Scoring is accuracy. | Expert | Checked automatically | 84% | System cards only |
| MedQA Taiwan Models answer Taiwan medical licensing exam questions in traditional Chinese. Scoring is accuracy. | Expert | Checked automatically | 91% | System cards only |
| MedQA USMLE (5 options) Models answer USMLE-style medical licensing questions with five answer options. Scoring is accuracy. | Expert | Checked automatically | 86% | System cards only |
| MMLU Anatomy Models answer MMLU multiple-choice questions from the Anatomy subject. Scoring is accuracy. | University | Checked automatically | 89% | System cards only |
| MMLU Clinical Knowledge Models answer MMLU multiple-choice questions from the Clinical Knowledge subject. Scoring is accuracy. | Expert | Checked automatically | 92% | System cards only |
| MMLU College Medicine Models answer MMLU multiple-choice questions from the College Medicine subject. Scoring is accuracy. | University | Checked automatically | 84% | System cards only |
| MMLU Medical Genetics Models answer MMLU multiple-choice questions from the Medical Genetics subject. Scoring is accuracy. | Expert | Checked automatically | 96% | System cards only |
| MMLU Professional Medicine Models answer MMLU multiple-choice questions from the Professional Medicine subject. Scoring is accuracy. | Expert | Checked automatically | 94% | System cards only |
| USMLE Step 1 (sample questions) Models answer the official USMLE Step 1 multiple choice sample questions. Scored by accuracy. | Expert | Checked automatically | 68.9% | System cards only |
| USMLE Step 2 (sample questions) Models answer the official USMLE Step 2 multiple choice sample questions. Scored by accuracy. | Expert | Checked automatically | 63.3% | System cards only |
| USMLE Step 3 (sample questions) Models answer the official USMLE Step 3 multiple choice sample questions. Scored by accuracy. | Expert | Checked automatically | 67.2% | System cards only |
| AfriMed-QA Models answer about 15,000 open and multiple-choice medical questions from over 60 medical schools in 16 African countries. Questions span 32 specialties. | University | Checked automatically | Not in cards | |
| AgentClinic Doctor agents interview simulated patients, order tests and reach a diagnosis in multimodal clinical scenarios built from MedQA and NEJM cases. | Expert | Checked automatically | Not in cards | |
| Bedside Bench Models answer 500 physician-written clinical cases on diagnosis, workup, escalation and medication safety. An LLM judge grades each answer against the authors' criteria. | Expert | Graded by a model | Not in cards | |
| Chest ImaGenome Models draw bounding boxes around named anatomical regions in MIMIC-CXR chest X-rays. Scored by mean intersection over union. | Expert | Checked automatically | Not in cards | |
| ChestX-ray14 Models read frontal chest X-rays from the NIH Clinical Center and say whether thoracic conditions are present. | Expert | Checked automatically | Not in cards | |
| CheXpert Models read chest radiographs and label up to 14 observations. The test labels come from board-certified radiologists. | Expert | Checked automatically | Not in cards | |
| CHI-Bench Agents run long healthcare operations workflows such as prior authorization through 87 MCP tools over 20 simulated apps. Tasks need policy lookups, role handoffs and multi-turn dialogs. | Expert | Checked automatically | Not in cards | |
| CliniCARE-Bench Agents answer a clinical question about one real MIMIC-IV patient through governed chart tools, then give a four-way verdict with cited evidence across 750 cases. | Expert | Graded by a model | Not in cards | |
| CT-RATE Models read 3D non-contrast chest CT volumes and say whether each of 18 conditions is present. Scored by macro accuracy or F1 against report-derived labels. | Expert | Checked automatically | Not in cards | |
| DAYJOB: Healthcare Agents handle short requests from healthcare staff using charts, claims exports, policies and emails. An agent judge applies an expert rubric, and a pass needs 95% of criteria. | Expert | Graded by a model | Not in cards | |
| EHRNoteQA Models read a patient's accumulated MIMIC-IV discharge summaries and answer clinical questions about them. The set has 962 question and answer pairs, each for a different patient. | Expert | Checked automatically | Not in cards | |
| EHRQA (Google) Models answer questions about long synthetic outpatient health records in FHIR format. Questions cover 42 types in 10 categories and need multi-hop, temporal and arithmetic reasoning. | Expert | Checked automatically | Not in cards | |
| EyePACS (diabetic retinopathy grading) Models grade diabetic retinopathy in a fundus photo on a 5-level scale, posed as a 5-option question. Google's test set has 3,614 images, one per patient. | Expert | Checked automatically | Not in cards | |
| GMAI-MMBench Vision-language models answer medical multiple-choice questions over many imaging modalities, clinical tasks and departments, at several perceptual granularities. | Expert | Checked automatically | Not in cards | |
| Health dynamic conversation eval Simulated users hide key health facts at first, and models must ask for context to reach the correct clinical conclusion. | Expert | Graded by a model | System cards only | |
| Health language and locale eval Models answer locale-specific health conversations in 10 languages across 25 locales, with search. Graded against clinician guidelines. | Expert | Graded by a model | System cards only | |
| Health static understanding eval Models answer static health questions over text conversations, documents such as lab reports, and images. Graded against clinician guidelines. | Expert | Graded by a model | System cards only | |
| MedAgentBench Agents use FHIR APIs in a virtual electronic health record to do 300 patient tasks in 10 categories written by physicians. | Expert | Checked automatically | Not in cards | |
| MedXpertQA MM Extended Models answer multimodal medical questions over clinical images and patient details. Mercor adds 51 new expert-written tasks to the 200 public ones and reports both scores. | Expert | Checked automatically | Not in cards | |
| MIMIC-CXR (classification) Models read chest X-rays from Beth Israel Deaconess and say whether five common conditions are present. Scored by average F1 over the conditions. | Expert | Checked automatically | Not in cards | |
| MIMIC-CXR report generation Models write the findings of a radiology report for a MIMIC-CXR chest X-ray. Scored against the radiologist's report, most often by RadGraph F1. | Expert | Checked automatically | Not in cards | |
| MindEval Models hold multi-turn therapy conversations with simulated patients designed with licensed clinical psychologists. An LLM judge scores each session. | Expert | Graded by a model | Not in cards | |
| MindEval2 Models support simulated patients from MindSim, a tool-using harness with detailed profiles. Psychologist-written rubrics (MHQR) replace Likert scales for scoring. | Expert | Graded by a model | Not in cards | |
| MLCR-AA Models reason over synthetic medical claim files of about 25k to 64k tokens, as a claims reviewer would. Three judge models check completeness and accuracy. | Expert | Graded by a model | Not in cards | |
| MMLU Medical Models answer the multiple-choice questions from the medical subjects of MMLU, such as anatomy, clinical knowledge and professional medicine. | University | Checked automatically | Not in cards | |
| MS-CXR-T Models compare a current and a prior chest X-ray of one patient and say whether a finding improved, stayed stable or got worse. | Expert | Checked automatically | Not in cards | |
| PathMCQA (Google) Models answer 4 to 9 option questions on histopathology patches about cancer identification, grading and subtype. The test split has 450 patches from 354 slides. | Expert | Checked automatically | Not in cards | |
| PatientAgentBench Patient-facing health agents talk with a simulated patient and use a sandbox of healthcare tools for scheduling, prescriptions, telehealth and escalation. An LLM jury scores over a hundred clinician-grounded criteria. | Expert | Graded by a model | Not in cards | |
| Radiology's Last Exam (RadLE) Multimodal models make a single spot diagnosis from images alone on 50 hard radiology cases. Board-certified radiologists scored about 83%. | Expert | Checked automatically | Not in cards | |
| US-DermMCQA (Google) Models pick the skin condition in a teledermatology image from 4 options. The set has one image per patient from 1,996 patients and 136 conditions. | Expert | Checked automatically | Not in cards | |
| VQA-RAD Vision-language models answer clinicians' questions about 315 radiology images, including head CTs, MRIs and chest X-rays. Questions are closed yes or no, or open free text. | Expert | Checked automatically | Not in cards |
Other professions
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Public Benefits Bench Models answer SNAP benefit questions across seven lifecycle phases and 25 topics. A judge grades each answer against rubrics validated by SNAP policy experts. | Expert | Graded by a model | 76.9% | Not in cards |
| FailureSensorIQ Models answer multiple-choice questions on how industrial sensors relate to failure modes of assets. The Kaggle board shows it under IBM's AssetOpsBench citation. | Expert | Checked automatically | 69.5% | Not in cards |
| Education (ByteDance Seed in-house) Models solve, grade, explain and write K to 12 problems across core subjects in teaching scenarios. | School | Graded by a model |  62.8% | Fading |
| Advanced Sommelier (theory knowledge) Models answer wine theory questions at the advanced sommelier level. Scoring is accuracy. | Expert | Checked automatically | 77% | System cards only |
| Certified Sommelier (theory knowledge) Models answer wine theory questions at the certified sommelier level. Scoring is accuracy. | Expert | Checked automatically | 86% | System cards only |
| Introductory Sommelier (theory knowledge) Models answer wine theory questions at the introductory sommelier level. Scoring is accuracy. | Expert | Checked automatically | 92% | System cards only |
| AssetOpsBench Agents automate industrial asset work, such as condition monitoring, maintenance planning and intervention scheduling, using sensor data, failure models and work orders. | Expert | Graded by a model | Not in cards | |
| IndustryBench Models answer 2,049 Chinese industrial procurement questions grounded in national standards and product records. Answers must respect regulated thresholds and safety clauses. | Expert | Checked automatically | Not in cards | |
| SnorkelManufacturing Agents make engineering, production, quality, supplier and safety decisions in a simulated discrete-parts plant across 200 tasks, a third of them CAD tasks. | Expert | Graded by a model | Not in cards |