Benchmark Almanac

OpenAI-Proof Q&A

GPT-5.1-Codex-Max 8.00% leads since Nov 2025 Only in system cards

Agents diagnose 20 real OpenAI research and engineering bottlenecks using code, logs and run artifacts. Scored as pass@1.

In-house OpenAI AI self-improvement evaluation. Each problem took an OpenAI team over a day.

Made by OpenAI Released Aug 2025 Labs' internal R&D tests Tasks kept private Last in a system card Apr 2026 Last model added Apr 2026
Level
Research frontier Researchers, on open problems or tasks that take days
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
9
Models scored in lab cards
9
System cards report it, from 1 lab

2 models share the top score of 8.00% among models released by Apr 2026

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

The newest model with a score here came out in Apr 2026, and nobody has reported a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 8.00 GPT-5.1-Codex-Max
  2. 8.00 GPT-5.2 Codex
  3. 7.50 GPT-5.4 Mini
  4. 6.00 GPT-5.3 Codex
  5. 4.16 GPT-5.4
  6. 3.00 GPT-5.2
  7. 2.00 GPT-5
  8. 1.70 GPT-5.5
  9. 1.00 GPT-5 mini
  1. 1 GPT-5.1-Codex-Max 8.00%
  2. 2 GPT-5.2 Codex 8.00%
  3. 3 GPT-5.4 Mini 7.50%
  4. 4 GPT-5.3 Codex 6.00%
  5. 5 GPT-5.4 4.16%
  6. 6 GPT-5.2 3.00%
  7. 7 GPT-5 2.00%
  8. 8 GPT-5.5 1.70%
  9. 9 GPT-5 mini 1.00%
#ModelScoreSettingDateSource
1 GPT-5.1-Codex-Max 8.00% Nov 2025 Model card
2 GPT-5.2 Codex 8.00% Dec 2025 Model card
3 GPT-5.4 Mini 7.50% Mar 2026 Model card
4 GPT-5.3 Codex 6.00% Feb 2026 Model card
5 GPT-5.4 4.16% Mar 2026 Model card
6 GPT-5.2 3.00% Dec 2025 Model card
7 GPT-5 2.00% Aug 2025 Model card
8 GPT-5.5 1.70% Apr 2026 Model card
9 GPT-5 mini 1.00% Aug 2025 Model card

2 models have held number one since its release in Aug 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Lab claims Not tested since Apr 2026
GPT-5.1-Codex-Max 8.00%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.1-Codex-Max GPT-5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.1-Codex-Max

Labs report it only in system cards

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.

9 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
23 Apr 2026 GPT-5.5 system card 1.7 OpenAI-Proof Q&A, pass@1, printed chart label (Figure 22, 9.1.3.4) AI R&D risk
17 Mar 2026 GPT-5.4 Mini system card 7.5 OpenAI-Proof Q&A v0 (OPQA), pass@1 (Table 20) AI R&D risk
5 Mar 2026 GPT-5.4 system card 4.16 OpenAI-Proof Q&A, pass@1, printed chart label (Figure 17, 5.1.3.3) AI R&D risk
5 Feb 2026 GPT-5.3 Codex system card 6 OpenAI-Proof Q&A, pass@1, printed chart label (Figure 9, 5.1.3.2) AI R&D risk
18 Dec 2025 GPT-5.2 Codex system card addendum 8 OpenAI-Proof Q&A, pass@1, printed chart label (Figure 12, 5.1.3.4) AI R&D risk
11 Dec 2025 GPT-5.2 system card addendum 3 OpenAI-Proof Q&A (OPQA), pass@1, no browsing, printed chart label (Figure 16) AI R&D risk
18 Nov 2025 GPT-5.1-Codex-Max system card 8 OpenAI-Proof Q&A, pass@1, printed chart label (8% also with refusals counted) (Figure 14, 5.1.3.5) AI R&D risk
7 Aug 2025 GPT-5 system card 2 OpenAI-Proof Q&A, pass@1, no browsing launch candidate (Figure 26) AI R&D risk
7 Aug 2025 GPT-5 mini system card 1 OpenAI-Proof Q&A, pass@1, gpt-5-thinking-mini launch candidate (Figure 26) AI R&D risk

More in AI research

All 83 in AI research
  • MLS-Bench In use Agent must invent one algorithmic change in an ML research scaffold whose gain transfers across settings. Scored relative to human baselines. Claude Fable 5.1 50.3%
  • PaperBench In use Agent replicates ICML 2024 papers from scratch. LLM judge grades against author-written hierarchical rubrics. Qwen 3.8 Max 93%
  • MLE-bench In use Kaggle-style ML engineering competitions. Agent trains models and is scored against competition leaderboards. GPT-6 Astra 93.8%
  • PostTrainBench 1.2 In use CLI agent gets a base model, one H100 and a time budget to post-train it for a target benchmark. Scored on normalized gain. Claude Fable 5.1 44.6%
  • KernelBench In use Replace PyTorch operators with faster custom GPU kernels. Scored on correctness and speedup. MiniMax-M3 28.8%
  • OpenAI PRs Only in system cards Agents replicate internal OpenAI pull requests from a pre-PR branch and a prompt. Graded by hidden unit tests as pass@1. GPT-5.2 55%