Benchmark Almanac

BioSecBench-Refusal

Grok 4.7 62.4% leads since Sep 2026 Only in system cards

Agents run 61 routine and 46 concealed-hazard biology analysis tasks. The score pairs red-team refusal with routine completion.

One of three BioSecBench evaluations. LatchBio reports a trial-weighted harmonic mean of red-team refusal and routine compliance.

In the authors’ words

As AI agents are incorporated into life science workflows, the capabilities that speed discovery might also enable misuse. We present BioSecBench-Refusal, a benchmark for risk identification and refusal behavior for biological research tasks. The benchmark pairs 61 Routine tasks, legitimate analyses adapted from the published literature, with 46 Red-Team tasks, fictional scenarios that resemble real research but conceal a biosecurity hazard. Across 16 model-harness configurations, refusal rates ranged from 7 percent to 74 percent on Routine tasks and 1 percent to 62 percent on Red-Team tasks, with many configurations refusing legitimate Routine work at comparable or higher rates than concealed hazards. Refusals were most often triggered by provider API filters applied prior to agentic reasoning. However, models given room to reason showed the potential to identify more real threats. We release BioSecBench-Refusal as a tool for model developers to calibrate capability and caution for agentic biotech research and development.

Abstract of BioSecBench-Refusal: A paired metric for performance and alignment in agentic biosecurity risk assessment (arXiv:2607.05462, CC0)
Made by LatchBio Released Jul 2026 Hazardous knowledge Last in a system card Sep 2026 Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
latchbio/biosecbench-refusal 1 stars Paper Leaderboard latch.bio
22
Models scored on its own leaderboard
1
System card reports it, from 1 lab

Grok 4.7 leads with 62.4%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.

22 models. Scroll sideways for the rest.

  1. 62.4 Grok 4.7
  2. 55.3 Muse Wonder
  3. 54.8 Gemini 3.7 Flash
  4. 48.7 Gemini 4 Argon
  5. 47.4 Gemini 3.8 Flash
  6. 46.9 Grok 4.6
  7. 40.8 GPT-6 Astra
  8. 37.9 GPT-6 Sol
  9. 37.6 GPT-6.1 Sol
  10. 37.5 GPT-6 Luna
  11. 34.9 Muse Spark Open
  12. 34.2 GPT-6.1 Terra
  13. 31.8 Claude Opus 5
  14. 11.3 DeepSeek V4.1 Flash
  15. 10.9 GLM-5.3
  16. 8.30 Kimi K3
  17. 5.60 Claude Fable 5.1
  18. 3.80 Claude Opus 5.5
  19. 3.40 Beam 1.0
  20. 3.40 Mistral Large 4
  21. 0.0 Nemotron 3 Ultra
  22. 0.0 Nemotron 3.5 Lightning 30B-A3B
  1. 1 Grok 4.7 62.4%
  2. 2 Muse Wonder 55.3%
  3. 3 Gemini 3.7 Flash 54.8%
  4. 4 Gemini 4 Argon 48.7%
  5. 5 Gemini 3.8 Flash 47.4%
  6. 6 Grok 4.6 46.9%
  7. 7 GPT-6 Astra 40.8%
  8. 8 GPT-6 Sol 37.9%
  9. 9 GPT-6.1 Sol 37.6%
  10. 10 GPT-6 Luna 37.5%
  11. 11 Muse Spark Open 34.9%
  12. 12 GPT-6.1 Terra 34.2%
  13. 13 Claude Opus 5 31.8%
  14. 14 DeepSeek V4.1 Flash 11.3%
  15. 15 GLM-5.3 10.9%
  16. 16 Kimi K3 8.30%
  17. 17 Claude Fable 5.1 5.60%
  18. 18 Claude Opus 5.5 3.80%
  19. 19 Beam 1.0 3.40%
  20. 20 Mistral Large 4 3.40%
  21. 21 Nemotron 3 Ultra 0%
  22. 22 Nemotron 3.5 Lightning 30B-A3B 0%
#ModelScoreSettingDateSource
1 Grok 4.7 62.4% Pi Sep 2026 Official leaderboard
2 Muse Wonder 55.3% Pi Oct 2026 Official leaderboard
3 Gemini 3.7 Flash 54.8% Pi Aug 2026 Official leaderboard
4 Gemini 4 Argon 48.7% Pi Sep 2026 Official leaderboard
5 Gemini 3.8 Flash 47.4% Pi Sep 2026 Official leaderboard
6 Grok 4.6 46.9% Pi Aug 2026 Official leaderboard
7 GPT-6 Astra 40.8% Pi Sep 2026 Official leaderboard
8 GPT-6 Sol 37.9% Codex Sep 2026 Official leaderboard
9 GPT-6.1 Sol 37.6% Pi Sep 2026 Official leaderboard
10 GPT-6 Luna 37.5% Codex Sep 2026 Official leaderboard
11 Muse Spark Open 34.9% Pi Oct 2026 Official leaderboard
12 GPT-6.1 Terra 34.2% Codex Oct 2026 Official leaderboard
13 Claude Opus 5 31.8% Claude Code Jul 2026 Official leaderboard
14 DeepSeek V4.1 Flash 11.3% Pi Sep 2026 Official leaderboard
15 GLM-5.3 10.9% Pi Aug 2026 Official leaderboard
16 Kimi K3 8.30% Pi Jul 2026 Official leaderboard
17 Claude Fable 5.1 5.60% pi Sep 2026 Official leaderboard
18 Claude Opus 5.5 3.80% Pi Sep 2026 Official leaderboard
19 Beam 1.0 3.40% Pi Oct 2026 Official leaderboard
20 Mistral Large 4 3.40% Pi Oct 2026 Official leaderboard
21 Nemotron 3 Ultra 0% Pi Jun 2026 Official leaderboard
22 Nemotron 3.5 Lightning 30B-A3B 0% Pi Aug 2026 Official leaderboard

Labs report scores for 1 model in their cards. We plot every source in the charts below.

6 models have held number one since its release in Jul 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

NVIDIA Moonshot AI Anthropic xAI Google DeepMind
Own leaderboard 6 leaders
Grok 4.7 62.4%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Grok 4.7 Gemini 3.7 Flash Grok 4.6 Claude Opus 5 Kimi K3 Nemotron 3 Ultra 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) Grok 4.7 Gemini 3.7 Flash

Labs report it only in system cards

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.

One system card reports it

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
21 Sep 2026 Grok 4.7 model card 62.4 BioSecBench Refusal, xhigh, Grok Build harness (7.1 table) Bio and chem risk

More in Biology and chemistry

All 96 in Biology and chemistry
  • SUPERChem In use Models solve 500 expert-written chemistry reasoning problems, offered in multimodal and text-only forms. Expert solution paths also allow a reasoning path score. Tencent Hy4 preview 66.4%
  • GeneBench Pro In use Agents analyze messy simulated genomics and biology datasets to estimate a decision-relevant quantity across 129 problems. Scored automatically against known answers. GPT-6 Astra 37.1%
  • LifeSciBench In use Models complete 750 expert-written life science research tasks, often with attached figures, tables, or sequence files. Graded against expert rubrics as score and pass rate. GPT-6 Astra 60.3%
  • MolecularIQ In use Models answer questions about molecular graphs, such as counting, indexing and constrained generation, and a symbolic checker verifies each answer. Intern-S2-397B 62.4%
  • TOMG-Bench In use Models generate molecules from natural language requests in three tasks: molecule editing, molecule optimization and customized generation. Each output is checked for validity and for meeting the requested properties. Intern-S2-397B 66.8%
  • BABE Fading Models reason from experimental results in peer-reviewed biology papers to draw conclusions, with causal and cross-scale inference. Seed2.0 Lite (0428) 57.9%