Biology and chemistry
We track 96 benchmarks in biology and chemistry, or 103 if you count each version. They run from competition level to research frontier level.
Anthropic held the top biology and chemistry score 44% of the contested time
That's about 20 of the 46 months since January 2023, averaged over the benchmarks below. The easiest benchmarks are at the top. Versions of one test share a row, with a dark tick at each new version. We stop the clock, and fade the rest of the row, when the best score passes 85%, or at the newest score if 45 days go by without another.
Google DeepMind OpenAI xAI Anthropic NVIDIA Moonshot AI Meta
Expert Specialists with years of training, often over hours of work
Research frontier Researchers, on open problems or tasks that take days
Share of contested time at number one
Anthropic 44% OpenAI 34% xAI 19% 
Google DeepMind 2% 
NVIDIA 1%
Google DeepMind 2% NVIDIA 1% Scores come from each benchmark’s own leaderboard, from Epoch AI’s runs and from labs’ reports on their own models. We date Epoch runs by the model’s release and lab reports by when they came out. Labs pick their own settings, so a lab’s claim can beat an independent run of the same model. Turn off lab claims to see independent runs only.
Google DeepMind OpenAI Anthropic NVIDIA Moonshot AI xAI
Expert Specialists with years of training, often over hours of work
Research frontier Researchers, on open problems or tasks that take days
Share of contested time at number one, independent runs only
Anthropic 71% OpenAI 21% xAI 4% Google DeepMind 3% NVIDIA 1%
Google DeepMind 3% NVIDIA 1% Only each benchmark’s own leaderboard and Epoch AI’s runs, dated by the model’s release. We leave out benchmarks that only labs report.
From competition to research frontier level
Grouped by who could solve the tasks, from things most adults can do to open research problems. The bar under each name is its best score.
Expert
Specialists with years of training, often over hours of work
- Virology Capabilities Test (VCT)
- OncologyBench
- Tacit Knowledge and Troubleshooting
- TxBench-Antibody-Discovery
- TxBench-Oligo-Discovery
- TxBench-Preclinical-Pharmacology
- BioSecBench-Function
- BioSecBench-Surveillance
- EpiBench
- scBench-Long
- SpatialBench-Long
- VariantBench
- MetagenomicsBench
- Multimodal Troubleshooting Virology
- ProtocolQA Open-Ended
- BioSecBench-Refusal
- DrugDiscoveryBench
- TroubleshootingBench
- LAB-Bench ProtocolQA
- LAB-Bench SeqQA
- LAB-Bench CloningScenarios
- LAB-Bench FigQA
- BioLP-bench
- SUPERChem
- BiomniBench
- GeneBench Pro
- Human Pathogen Capabilities Test
- LifeSciBench
- Creative biology
- Molecular Biology Capabilities Test
- SMDD-Bench
- WMDP Biology
- DNA Synthesis Screening Evasion
- WMDP Chemistry
- ABC Bench Liquid-Handling Robot
- BABE
- Biology-Instructions
- GeneBench
- Meta BioDesign Proxy
- Meta chemical-agent refusal evaluation
- Mol-Instructions
- SecureBio VMQA, single choice
- SMolInstruct
- ABC Bench Fragment Design
- ASL-4 virology uplift trial (Anthropic)
- Bioweapons acquisition uplift trial (Anthropic)
- Bioweapons knowledge questions
- LAB-Bench
- Long-form Biological Risk Questions
- ProtocolQA Open-Ended (xAI)
- Protocols (Anthropic life sciences)
- WMDP
- ABC Bench Screening Evasion
- Biochemistry knowledge improvement
- Biosecurity VCT
- BioUseBench
- MolecularIQ
- ProteinLMBench
- ReproBAIT
- Tacit Knowledge Brainstorm (Open-Ended)
- TOMG-Bench
- AAV capsid packaging prediction
- Bioinformatics evaluations (Anthropic)
- Biological static hazard evaluation
- Creative Biology Uplift Trial (Anthropic)
- Long-form virology tasks
- Meta BioProtocol Proxy
- VCT-v2
- Virology Capabilities Test (text-only)
- AAV packaging rate auto-research evaluation
- Biological threat creation uplift study
- BioRiskEval
- Biotool and Wet Lab Actions
- Black-box RNA sequence modeling and design
- Catastrophic biology scenario uplift trial (Anthropic)
- CBRN expert red teaming
- CBRN open-ended questions
- Expert comparisons on biothreat information
- LAB-Bench LitQA2
- LABBench2 FigQA2
- LABBench2 LitQA3
- LABBench2 ProtocolQA2
- LABBench2 SeqQA2
- MedChemBench (OpenAI)
- Meta BioKnowledge Proxy
- Meta chemical and biological high-risk bottleneck evaluations
- Short-horizon computational biology tasks
- SoSBench
- VIALS
Research frontier
Researchers, on open problems or tasks that take days
- Human Pathogen Capabilities Test v2 (HPCT v2)
- Agentic BAIM-LLM Evaluation (ABLE)
- AAV Capsid Packaging Prediction (OpenAI)
- Coronavirus-ACE2 Cell-Entry Screen
- Phage-plasmid Co-evolution
- SHP2 Protein Function Prediction
- DNA sequence design for transcription factor binding
- Hard-negative protein binding prediction
- World-Class Bio
- Advanced Screening Evasion
- Structured expert probing campaign: chem-bio novel design
Another 2 have no level. Levels don’t apply to arenas, safety evaluations or composite indices.
By kind
Lab protocols
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Tacit Knowledge and Troubleshooting Models answer multiple-choice questions on tacit lab knowledge and troubleshooting that need hands-on expertise. Scored as cons@32. | Expert | Checked automatically | 94% | System cards only |
| Multimodal Troubleshooting Virology Models troubleshoot virology lab scenarios, sometimes with images, and must select every correct answer. Scored as refusal-adjusted pass@1. | Expert | Checked automatically | 59% | System cards only |
| ProtocolQA Open-Ended Models correct errors in published lab protocols using open-ended questions adapted from LAB-Bench ProtocolQA. Scored as pass@1. | Expert | Graded by a model | 44.4% | System cards only |
| TroubleshootingBench Models find and correct realistic experimental errors in expert-written wet lab protocols. Scored as pass@1. | Expert | Checked automatically | 49.6% | System cards only |
| LAB-Bench ProtocolQA Models find and fix errors in lab protocols in multiple-choice form. ProtocolQA subset of LAB-Bench. | Expert | Checked automatically | 90.7% | System cards only |
| LAB-Bench CloningScenarios Models answer multiple-choice questions about multi-step molecular cloning workflows. CloningScenarios subset of LAB-Bench. | Expert | Checked automatically | 94.2% | System cards only |
| BioLP-bench Find the introduced error in a biological lab protocol. Tests protocol understanding. | Expert | Graded by a model | 49% | System cards only |
| ProtocolQA Open-Ended (xAI) Models name the single most important mistake in a biology lab protocol in free text, with no answer choices. | Expert | Graded by a model | 87% | System cards only |
| Protocols (Anthropic life sciences) Models find and fix errors in molecular biology protocols and answer practical questions about applying real laboratory protocols. | Expert | Checked automatically | 62% | System cards only |
| Tacit Knowledge Brainstorm (Open-Ended) Models answer 50 open-ended troubleshooting questions drawn from virologists' and molecular biologists' careers. Scored as pass@1 against an ideal answer. | Expert | Graded by a model | 51% | System cards only |
| Meta BioProtocol Proxy Models answer 400 questions on sequence prediction, correction and missing steps in 60 expert protocols for 15 low-risk proxy agents. | Expert | Checked automatically |  43.6% | System cards only |
| VCT-v2 Models answer an updated set of expert virology troubleshooting questions from SecureBio. Scored as refusal-corrected accuracy. | Expert | Checked automatically | 57.7% | System cards only |
| Virology Capabilities Test (text-only) Models answer the text-only questions of the SecureBio Virology Capabilities Test. Scored as accuracy. | Expert | Checked automatically | 46% | System cards only |
Genomics and life sciences
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| OncologyBench Separate agents run analyses of cancer research data, and a final agent makes a decision on detection, diagnosis or treatment. Strict scoring needs full marks on both parts. | Expert | Graded by a model | 50.7% | Not in cards |
| TxBench-Antibody-Discovery Agents make antibody discovery and development decisions from real experimental data. A grader checks each decision. | Expert | Checked automatically | 67.7% | Not in cards |
| TxBench-Oligo-Discovery Agents make oligonucleotide discovery decisions, such as ASO and siRNA design, from real data. A grader checks each decision. | Expert | Checked automatically | 62.2% | Not in cards |
| TxBench-Preclinical-Pharmacology Agents analyze preclinical pharmacology data and make development decisions. A grader checks each decision. | Expert | Checked automatically | 65.3% | Not in cards |
| BioSecBench-Function Agents recover biological function from experimental data. Scored by verifiable checks. One of three LatchBio BioSecBench evaluations. | Expert | Checked automatically | 73.6% | System cards only |
| BioSecBench-Surveillance Agents turn raw pathogen sequencing data into surveillance results. The endpoint pass rate excludes refused runs. | Expert | Checked automatically | 60.8% | System cards only |
| EpiBench Agents analyze real epigenomics datasets in a sandbox with standard tools. A deterministic grader checks the biological result. | Expert | Checked automatically | 39.6% | Not in cards |
| scBench-Long Agents run long single-cell RNA-seq analyses that need sustained empirical work and biological reasoning. | Expert | Checked automatically | 36.9% | Not in cards |
| SpatialBench-Long Agents run long, multi-step analyses of real spatial biology datasets that need sustained biological reasoning. | Expert | Checked automatically | 40.5% | Not in cards |
| VariantBench Agents discover and interpret genetic variants from real sequencing analyses. A deterministic grader checks the result. | Expert | Checked automatically | 52.8% | Not in cards |
| MetagenomicsBench Agents analyze microbial communities from shotgun, amplicon, long-read and spatial metagenomics data. A grader checks the biological result. | Expert | Checked automatically | 62.7% | Not in cards |
| DrugDiscoveryBench Agents do early-stage drug discovery work, from target identification to lead optimization, in 82 tasks with one checkable answer each. A judge model grades answers against expert rubrics. | Expert | Graded by a model | 68.7% | Not in cards |
| LAB-Bench SeqQA Models answer multiple-choice questions that require DNA and protein sequence manipulation. SeqQA subset of LAB-Bench. | Expert | Checked automatically | 98.2% | System cards only |
| LAB-Bench FigQA Models interpret figures from biology papers and answer multiple-choice questions. FigQA subset of LAB-Bench. | Expert | Checked automatically | 80.4% | System cards only |
| SUPERChem Models solve 500 expert-written chemistry reasoning problems, offered in multimodal and text-only forms. Expert solution paths also allow a reasoning path score. | Expert | Checked automatically |  66.4% | In use |
| BiomniBench Agents redo data analysis tasks from 100 published biomedical studies in a Harbor environment. An LLM judge grades the whole trajectory against rubrics written with the paper authors, so method quality counts, not just the final answer. | Expert | Graded by a model | 63.9 | Not in cards |
| GeneBench Pro Agents analyze messy simulated genomics and biology datasets to estimate a decision-relevant quantity across 129 problems. Scored automatically against known answers. | Expert | Checked automatically | 37.1% | In use |
| LifeSciBench Models complete 750 expert-written life science research tasks, often with attached figures, tables, or sequence files. Graded against expert rubrics as score and pass rate. | Expert | Graded by a model | 60.3% | In use |
| Molecular Biology Capabilities Test Models answer expert-level molecular biology questions from SecureBio. Scored as accuracy. | Expert | Checked automatically | 60% | System cards only |
| SMDD-Bench 502 small-molecule drug-design tasks in 5 types. Agent uses a fixed toolkit to produce a molecule that must pass all task criteria. | Expert | Checked automatically | 40.2% | Not in cards |
| AAV Capsid Packaging Prediction (OpenAI) Models predict packaging scores for held-out AAV capsid variants. Scored as Spearman rank correlation. | Research frontier | Checked automatically | 0.53 | System cards only |
| Coronavirus-ACE2 Cell-Entry Screen Models predict and rank held-out cell entry measurements for spike protein and ACE2 pairings. Scored as a composite score. | Research frontier | Checked automatically | 0.47 | System cards only |
| Phage-plasmid Co-evolution Models predict future amino acid mutations in a phage and plasmid system from earlier observations. Scored as negative log likelihood. | Research frontier | Checked automatically | 10.7 | System cards only |
| SHP2 Protein Function Prediction Models predict held-out assay measurements for single amino acid variants of SHP2. Scored as mean R squared over three assay datasets. | Research frontier | Checked automatically | 0.40 | System cards only |
| BABE Models reason from experimental results in peer-reviewed biology papers to draw conclusions, with causal and cross-scale inference. | Expert | Graded by a model |  57.9% | Fading |
| Biology-Instructions Models answer multi-omics sequence tasks on DNA, RNA, protein and multi-molecule inputs, such as property prediction and interaction tasks. Scored per task with task metrics. | Expert | Checked automatically |  55.7% | Fading |
| DNA sequence design for transcription factor binding Models design DNA sequences that bind 11 Nucleobench transcription factors, scored by BPNet oracles against the Ledidi baseline. Scored as pass@1. | Research frontier | Checked automatically | 16.5% | System cards only |
| GeneBench Agents analyze messy genomics and quantitative biology data across 103 multi-step problems. Scored by pass rate on verifiable final answers. | Expert | Checked automatically | 33.2% | Fading |
| Hard-negative protein binding prediction Agents with structure files and tools pick the one true binder among five similar designed proteins. Scored as pass@4. | Research frontier | Checked automatically | 7.60% | System cards only |
| Meta BioDesign Proxy Agents complete computational biology design tasks across host-pathogen interaction, protein design and sequence analysis. Aggregate normalized score. | Expert | Checked automatically | 79.4 | System cards only |
| Mol-Instructions Models answer molecule-oriented, protein-oriented and biomolecular text instructions, such as reaction prediction, molecule description and protein function questions. Scored per task with task metrics. | Expert | Checked automatically | 54% | Fading |
| SMolInstruct Models solve 14 chemistry tasks such as name conversion, property prediction, molecule captioning, generation and reaction prediction. Each task uses its own metric on the test split. | Expert | Checked automatically | 74.8% | Fading |
| BIOBench (ByteDance Seed) Models answer 150 olympiad-level biology questions, 80% of them with images. Domain experts annotated and reviewed each question. | Competition | Checked automatically | 53.5% | Fading |
| LAB-Bench Multiple-choice tasks on practical biology research skills: literature, protocols, sequence and figure reasoning, database lookup. | Expert | Checked automatically | 80.7% | System cards only |
| World-Class Bio Models answer expert-level biology questions in a SecureBio benchmark. Scored as accuracy. | Research frontier | Checked automatically | 68.3% | System cards only |
| Biochemistry knowledge improvement Models answer biochemistry tasks on structures, mutation thermodynamics, reaction products, arrow pushing and DNA phylogeny. Scored as reward@4. | Expert | Checked automatically | 39.3% | System cards only |
| MolecularIQ Models answer questions about molecular graphs, such as counting, indexing and constrained generation, and a symbolic checker verifies each answer. | Expert | Checked automatically | 62.4% | In use |
| ProteinLMBench Models answer 944 hand-checked multiple choice questions on protein sequence, structure and function. | Expert | Checked automatically | 63.1% | Dropped |
| ReproBAIT Agents independently reproduce biological AI models from published scientific papers. Scored as success rate. | Expert | Checked automatically | 85% | System cards only |
| TOMG-Bench Models generate molecules from natural language requests in three tasks: molecule editing, molecule optimization and customized generation. Each output is checked for validity and for meeting the requested properties. | Expert | Checked automatically | 66.8% | In use |
| Bioinformatics evaluations (Anthropic) Agents process raw sequencing data with workflows from published papers and answer expert questions about the results. | Expert | Checked automatically | 73.7% | System cards only |
| LAB-Bench LitQA2 Models answer multiple-choice questions whose answers appear once in recent biology papers. Built to need literature search rather than recall. A LAB-Bench subtask. | Expert | Checked automatically | Not in cards | |
| LABBench2 FigQA2 Models read scientific figures and answer in open response. Served as images or as PDFs. The FigQA2 subtask of LABBench2. | Expert | Checked automatically | Not in cards | |
| LABBench2 LitQA3 Models answer open-response questions whose answers appear once in recent biology papers. The LitQA3 subtask of LABBench2. | Expert | Checked automatically | Not in cards | |
| LABBench2 ProtocolQA2 Models find and fix errors planted in published lab protocols, answering in open response. The ProtocolQA2 subtask of LABBench2. | Expert | Checked automatically | Not in cards | |
| LABBench2 SeqQA2 Models reason about DNA and protein sequences and answer in open response. The SeqQA2 subtask of LABBench2. | Expert | Checked automatically | Not in cards | |
| MedChemBench (OpenAI) Models answer chemistry questions from an OpenAI internal set, grouped with OpenAI's life science research evaluations. OpenAI has not published the task format. | Expert | Checked automatically | Not in cards | |
| VIALS Vision models answer 161 questions about lab images such as gel blots, plasmid maps, microscopy and flow cytometry plots. An LLM judge grades each answer against a key. | Expert | Graded by a model | Not in cards |
Hazardous knowledge
| Benchmark | Level | Scored | Best score | Status |
|---|---|---|---|---|
| Human Pathogen Capabilities Test v2 (HPCT v2) Models answer multiple-response questions on a small set of weaponizable pathogens that biosecurity experts rank highest priority. A revised version of HPCT. | Research frontier | Checked automatically | 80% | Not in cards |
| Virology Capabilities Test (VCT) Models answer 322 multimodal virology troubleshooting questions written by expert virologists. Multiple-select variant requires all correct options. | Expert | Checked automatically | 67.4% | System cards only |
| BioSecBench-Refusal Agents run 61 routine and 46 concealed-hazard biology analysis tasks. The score pairs red-team refusal with routine completion. | Expert | Graded by a model | 62.4% | System cards only |
| Agentic BAIM-LLM Evaluation (ABLE) Agents run a viral-protein redesign workflow. Nine tasks cover structure retrieval, design strategy, tool selection and biological AI model use. Outputs are auto-scored. | Research frontier | Checked automatically | 84.6% | Not in cards |
| Human Pathogen Capabilities Test Models answer expert-level questions on human pathogens from SecureBio. Scored as accuracy. | Expert | Checked automatically | 68.4% | System cards only |
| Creative biology Models answer questions about unusual biology needed to engineer harmless biological agents, a weak proxy for novel weapon design. | Expert | Checked automatically | 60.3% | System cards only |
| WMDP Biology Models answer 1,273 multiple-choice questions that proxy hazardous biosecurity knowledge. Biology split of WMDP. | Expert | Checked automatically | 90% | System cards only |
| DNA Synthesis Screening Evasion Models design DNA fragments that assemble into pathogenic viruses while evading gene synthesis screening. Scored per assembly criterion. | Expert | Checked automatically | 80% | System cards only |
| WMDP Chemistry Models answer 408 multiple-choice questions that proxy hazardous chemical security knowledge. Chemistry split of WMDP. | Expert | Checked automatically | 84.8% | System cards only |
| ABC Bench Liquid-Handling Robot Agents use biology tools to plan liquid-handling robot steps for a construct-assembly task. One task family of the Agentic Bio-Capabilities Benchmark. | Expert | Checked automatically | 100.0 | System cards only |
| Meta chemical-agent refusal evaluation Models receive high-risk chemical prompts and the benchmark measures how often they refuse. Covers development, procurement and production stages. | Expert | Graded by a model | 100% | System cards only |
| SecureBio VMQA, single choice Models answer 350 virology troubleshooting questions in single-choice form. VMQA is an earlier version of the Virology Capabilities Test. | Expert | Checked automatically | 61.2% | System cards only |
| ABC Bench Fragment Design Agents use biology tools to complete a computational genetic-construct design task. One task family of the Agentic Bio-Capabilities Benchmark. | Expert | Checked automatically | 97.0 | System cards only |
| ASL-4 virology uplift trial (Anthropic) PhD experts with model access write a protocol to reconstruct a challenging virus. Graded by a rubric with critical failures. | Expert | Graded by people | 66.8% | System cards only |
| Autointent-Bio Models face manual and synthetic dangerous and benign biology queries. xAI reports refusal recall on the dangerous ones. | Graded by a model | 100% | System cards only | |
| Autointent-Chem Models face manual and synthetic dangerous and benign chemistry queries. xAI reports refusal recall on the dangerous ones. | Graded by a model | 100% | System cards only | |
| Bioweapons acquisition uplift trial (Anthropic) Participants draft a bioweapons acquisition plan over two days with or without model access. Score is the uplift ratio of model group over control. | Expert | Graded by people | 2.53 | System cards only |
| Bioweapons knowledge questions Models answer 33 sensitive bioweapons questions written by Deloitte biosecurity staff. Answers are compared with novice, intermediate and expert baselines. | Expert | Graded by people | 51.5% | System cards only |
| Long-form Biological Risk Questions Models answer long-form questions across five biothreat creation stages, graded against expert rubrics. Scored as pass@1 per stage. | Expert | Graded by a model | 0% | System cards only |
| WMDP Proxy multiple-choice benchmark for hazardous knowledge in bio, chem and cyber. Released with the RMU unlearning method. | Expert | Checked automatically | 84.8% | System cards only |
| ABC Bench Screening Evasion Agents complete a computational task on nucleic-acid synthesis screening. One task family of the Agentic Bio-Capabilities Benchmark. | Expert | Checked automatically | 63.2 | System cards only |
| Biosecurity VCT Models answer dual-use virus lab troubleshooting items. Accuracy counts should-refuse and should-comply items together. | Expert | Checked automatically | 47.8% | System cards only |
| BioUseBench Models face dual-use biology enablement prompts. The reported metric is the refusal rate on the severity 5 subset. | Expert | Graded by a model | 91.4% | System cards only |
| AAV capsid packaging prediction Models give the probability that each of 1,000 unpublished modified AAV capsid sequences assembles into a functional capsid. Scored as AUROC. | Expert | Checked automatically | 0.80 | System cards only |
| Advanced Screening Evasion Models propose strategies to evade DNA synthesis screening and design evading fragments. Scored from 0 to 1 on strategy quality and fragment evasion. | Research frontier | Graded by people | 0.73 | System cards only |
| Biological static hazard evaluation Models answer 75 questions across priority biological threat scenarios, each asked 10 times. The score is the share of hazardous responses. | Expert | Graded by a model | 0.13% | System cards only |
| Creative Biology Uplift Trial (Anthropic) Participants with molecular biology PhDs write reports on novel biological workflows with or without model access. Expert graders score each report. | Expert | Graded by people | 16.4 | System cards only |
| Long-form virology tasks Agents complete end-to-end pathogen acquisition workflows, covering workflow design and laboratory protocols. Scored against expert criteria. | Expert | Graded by people | 69.7% | System cards only |
| AAV packaging rate auto-research evaluation Agents build predictors of AAV capsid packaging that must generalize to an unseen insertion region, using a limited scorer. Scored by Spearman correlation. | Expert | Checked automatically | System cards only | |
| Biological threat creation uplift study Experts and novices answer biothreat creation tasks with internet access alone or with model help. Scored as pass rates per stage. | Expert | Graded by people | System cards only | |
| BioRiskEval Researchers measure biorisk-relevant capability kept by open-weight DNA and protein foundation models, with three task sets named Gen, Mut and Vir. | Expert | Checked automatically | Not in cards | |
| Biotool and Wet Lab Actions Agents perform wet lab assistant tasks such as finding a genome file or ordering DNA through a computer or browser. Scored as pass@10 per task. | Expert | Checked automatically | System cards only | |
| Black-box RNA sequence modeling and design Models predict scores for unlabeled RNA sequences from labeled data, then design new sequences. Results are compared with 57 human experts. | Expert | Checked automatically | System cards only | |
| Catastrophic biology scenario uplift trial (Anthropic) PhD biologists with model access spend up to 16 hours planning a catastrophic biological scenario. External experts grade uplift and executability. | Expert | Graded by people | System cards only | |
| CBRN expert red teaming Experts try to extract uplift beyond a web baseline across staged CBRN attack scenarios. | Expert | Graded by people | System cards only | |
| CBRN open-ended questions Experts review model answers to single and multi-turn CBRN threat scenario questions for accuracy, novelty and actionable uplift. | Expert | Graded by people | System cards only | |
| Expert comparisons on biothreat information PhD biology experts compare model answers to expert answers on long-form biorisk questions. Scored as win rate on accuracy, understanding and ease of execution. | Expert | Graded by people | System cards only | |
| Meta BioKnowledge Proxy Models answer tacit knowledge and troubleshooting questions on wet-lab workflows for proxy biological agents. Has 200 single-answer and 100 multiple-answer questions. | Expert | Checked automatically | System cards only | |
| Meta chemical and biological high-risk bottleneck evaluations Models answer expert-written tasks in three restricted subsets. An expert-validated rubric scores each response. | Expert | Graded by a model | System cards only | |
| Short-horizon computational biology tasks Agents perform end-to-end computational tasks related to modifying and engineering biological agents. | Expert | Checked automatically | System cards only | |
| SoSBench Models answer hazardous requests drawn from real regulations in six scientific fields. Scored by how often the model complies with a request it should refuse. | Expert | Graded by a model | Not in cards | |
| Structured expert probing campaign: chem-bio novel design Biosecurity and chemistry experts test whether a model helps design novel chem-bio threats. Scenarios are rated against Preparedness risk levels. | Research frontier | Graded by people | System cards only |