Benchmark Almanac

TroubleshootingBench

GPT-5.5 Pro 49.6% leads since Apr 2026 Only in system cards

Models find and correct realistic experimental errors in expert-written wet lab protocols. Scored as pass@1.

In-house OpenAI evaluation. The tasks are not public.

Made by OpenAI Released Aug 2025 Lab protocols Tasks kept private Last in a system card Oct 2026 Last model added Oct 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Short answer A number, word or short string matched to a key.
19
Models scored in lab cards
28
System cards report it, from 1 lab

GPT-5.5 Pro leads with 49.6%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

19 models. Scroll sideways for the rest.

  1. 49.6 GPT-5.5 Pro
  2. 48.4 GPT-6 Astra
  3. 48.0 GPT-5.6 Sol
  4. 48.0 GPT-6.1 Sol
  5. 45.3 GPT-6 Sol
  6. 44.1 GPT-5.5
  7. 41.5 GPT-6 Luna
  8. 37.7 GPT-5.6 Luna
  9. 35.8 GPT-5.4
  10. 35.0 GPT-5.2
  11. 34.0 GPT-5.2 Codex
  12. 33.7 GPT-5.5 Instant
  13. 32.0 GPT-5.3 Codex
  14. 31.9 GPT-5.4 Mini
  15. 29.0 GPT-5.1-Codex-Max
  16. 26.0 GPT-5 mini
  17. 25.0 gpt-oss-120b
  18. 22.0 GPT-5
  19. 19.0 gpt-oss-20b
  1. 1 GPT-5.5 Pro 49.6%
  2. 2 GPT-6 Astra 48.4%
  3. 3 GPT-5.6 Sol 48%
  4. 4 GPT-6.1 Sol 48%
  5. 5 GPT-6 Sol 45.3%
  6. 6 GPT-5.5 44.1%
  7. 7 GPT-6 Luna 41.5%
  8. 8 GPT-5.6 Luna 37.7%
  9. 9 GPT-5.4 35.8%
  10. 10 GPT-5.2 35%
  11. 11 GPT-5.2 Codex 34%
  12. 12 GPT-5.5 Instant 33.7%
  13. 13 GPT-5.3 Codex 32%
  14. 14 GPT-5.4 Mini 31.9%
  15. 15 GPT-5.1-Codex-Max 29%
  16. 16 GPT-5 mini 26%
  17. 17 gpt-oss-120b 25%
  18. 18 GPT-5 22%
  19. 19 gpt-oss-20b 19%
#ModelScoreSettingDateSource
1 GPT-5.5 Pro 49.6% Apr 2026 Model card
2 GPT-6 Astra 48.4% Sep 2026 Model card
3 GPT-5.6 Sol 48% Jun 2026 Model card
4 GPT-6.1 Sol 48% Sep 2026 Model card
5 GPT-6 Sol 45.3% Sep 2026 Model card
6 GPT-5.5 44.1% Apr 2026 Model card
7 GPT-6 Luna 41.5% Oct 2026 Model card
8 GPT-5.6 Luna 37.7% Aug 2026 Model card
9 GPT-5.4 35.8% Mar 2026 Model card
10 GPT-5.2 35% Dec 2025 Model card
11 GPT-5.2 Codex 34% Dec 2025 Model card
12 GPT-5.5 Instant 33.7% May 2026 Model card
13 GPT-5.3 Codex 32% Feb 2026 Model card
14 GPT-5.4 Mini 31.9% Mar 2026 Model card
15 GPT-5.1-Codex-Max 29% Nov 2025 Model card
16 GPT-5 mini 26% Aug 2025 Model card
17 gpt-oss-120b 25% Aug 2025 Model card
18 GPT-5 22% Aug 2025 Model card
19 gpt-oss-20b 19% Aug 2025 Model card

6 models have held number one since its release in Aug 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Lab claims 6 leaders
GPT-5.5 Pro 49.6%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) GPT-5.5 Pro GPT-5.4 GPT-5.1-Codex-Max GPT-5 mini 0% 20% 40% 60% 80% 100% 2026 2027 Near the ceiling (85%) GPT-5.5 Pro GPT-5.4

Labs report it only in system cards

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.

28 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
29 Oct 2026 GPT-6 Luna system card addendum 37.9 TroubleshootingBench, pass@1 (Table 8) Bio and chem risk
7 Oct 2026 GPT-6 Sol system card addendum 41.8 TroubleshootingBench, pass@1 (Table 8) Bio and chem risk
7 Oct 2026 GPT-6 Luna system card addendum 41.5 TroubleshootingBench, pass@1 (Table 8) Bio and chem risk
29 Sep 2026 GPT-6.1 Sol system card addendum 48 TroubleshootingBench, refusal-adjusted pass@1 (9.1.1.1) Bio and chem risk
22 Sep 2026 GPT-6 Sol system card 45.3 TroubleshootingBench, pass@1 (Table 33, A.8.1.1.1) Bio and chem risk
22 Sep 2026 GPT-6 Luna system card 38.6 TroubleshootingBench, pass@1 (Table 33, A.8.1.1.1) Bio and chem risk
3 Sep 2026 GPT-6 Astra system card 48.4 TroubleshootingBench, pass@1 (refusal-adjusted 63.46%) (10.1.1.1.4) Bio and chem risk
6 Aug 2026 GPT-5.6 Sol system card addendum 47.6 TroubleshootingBench, pass@1 with refusals or safe completions counted (46.4% without), printed chart label (Figure 6, 7.1.1.4) Bio and chem risk
6 Aug 2026 GPT-5.6 Luna system card addendum 37.7 TroubleshootingBench, pass@1 with refusals or safe completions counted (37.5% without), printed chart label (Figure 6, 7.1.1.4) Bio and chem risk
9 Jul 2026 GPT-5.6 Terra system card TroubleshootingBench, pass@1 by cost, chart without printed numbers (Figures 28, 29, 9.1.1.4) Bio and chem risk
9 Jul 2026 GPT-5.6 Sol system card 48 TroubleshootingBench, pass@1, top score stated in text (9.1.1.4) Bio and chem risk
9 Jul 2026 GPT-5.6 Luna system card TroubleshootingBench, pass@1 by cost, chart without printed numbers (Figures 28, 29, 9.1.1.4) Bio and chem risk
25 Jun 2026 GPT-5.6 Terra system card TroubleshootingBench, pass@1 by cost, chart without printed numbers (Figures 28, 29, 9.1.1.4) Bio and chem risk
25 Jun 2026 GPT-5.6 Sol system card 48 TroubleshootingBench, pass@1, top score stated in text (9.1.1.4) Bio and chem risk
25 Jun 2026 GPT-5.6 Luna system card TroubleshootingBench, pass@1 by cost, chart without printed numbers (Figures 28, 29, 9.1.1.4) Bio and chem risk
4 May 2026 GPT-5.5 Instant system card 33.7 TroubleshootingBench, pass@1 (33.91% counting refusals or safe completions), printed chart label (Figure 6, 8.1.1.4) Bio and chem risk
23 Apr 2026 GPT-5.5 system card 44.1 TroubleshootingBench, pass@1, printed chart label (Figure 15, 9.1.1.4) Bio and chem risk
23 Apr 2026 GPT-5.5 Pro system card 49.6 TroubleshootingBench, pass@1, GPT-5.5 Pro, printed chart label (Figure 15, 9.1.1.4) Bio and chem risk
17 Mar 2026 GPT-5.4 Mini system card 31.9 TroubleshootingBench, pass@1 (33.11% counting refusals) (Table 18) Bio and chem risk
5 Mar 2026 GPT-5.4 system card 35.8 TroubleshootingBench, pass@1 excluding refusals (38.85% counting refusals and safe completions) (Figure 11, 5.1.1.4) Bio and chem risk
5 Feb 2026 GPT-5.3 Codex system card 32 TroubleshootingBench, pass@1, printed chart label (Figure 4, 5.1.1.4) Bio and chem risk
18 Dec 2025 GPT-5.2 Codex system card addendum 34 TroubleshootingBench, pass@1, printed chart label (Figure 4, 5.1.1.4) Bio and chem risk
11 Dec 2025 GPT-5.2 system card addendum 35 TroubleshootingBench, pass@1, gpt-5.2 browsing, printed chart label (Figure 8) Bio and chem risk
18 Nov 2025 GPT-5.1-Codex-Max system card 29 TroubleshootingBench, pass@1, printed chart label (40% with refusals counted as successes) (Figure 5, 5.1.1.5) Bio and chem risk
7 Aug 2025 GPT-5 system card 22 TroubleshootingBench, pass@1, browsing launch candidate (helpful-only 33%) (Figure 13) Bio and chem risk
7 Aug 2025 GPT-5 mini system card 26 TroubleshootingBench, pass@1, gpt-5-thinking-mini browsing launch candidate (Figure 13) Bio and chem risk
5 Aug 2025 gpt-oss-20b model card 19 TroubleshootingBench, pass@1, gpt-oss 20b browsing launch candidate (Figure 9) Bio and chem risk
5 Aug 2025 gpt-oss-120b model card 25 TroubleshootingBench, pass@1, gpt-oss 120b browsing launch candidate (Figure 9) Bio and chem risk

More in Biology and chemistry

All 96 in Biology and chemistry
  • SUPERChem In use Models solve 500 expert-written chemistry reasoning problems, offered in multimodal and text-only forms. Expert solution paths also allow a reasoning path score. Tencent Hy4 preview 66.4%
  • GeneBench Pro In use Agents analyze messy simulated genomics and biology datasets to estimate a decision-relevant quantity across 129 problems. Scored automatically against known answers. GPT-6 Astra 37.1%
  • LifeSciBench In use Models complete 750 expert-written life science research tasks, often with attached figures, tables, or sequence files. Graded against expert rubrics as score and pass rate. GPT-6 Astra 60.3%
  • MolecularIQ In use Models answer questions about molecular graphs, such as counting, indexing and constrained generation, and a symbolic checker verifies each answer. Intern-S2-397B 62.4%
  • TOMG-Bench In use Models generate molecules from natural language requests in three tasks: molecule editing, molecule optimization and customized generation. Each output is checked for validity and for meeting the requested properties. Intern-S2-397B 66.8%
  • BABE Fading Models reason from experimental results in peer-reviewed biology papers to draw conclusions, with causal and cross-scale inference. Seed2.0 Lite (0428) 57.9%