Atomic Challenges

GPT-6 Astra 90.9% Only in system cards

Agents solve 22 skill-level offensive security challenges in vulnerability research, network attack simulation and evasion. Scored as challenges solved.

Run by Irregular. The irregular-frontiercyber-and-atomic-challenges record lumps it with FrontierCyber.

Made by Irregular Released Dec 2025 Capture-the-flag challenges Tasks kept private Last in a system card Sep 2026 Last model added Sep 2026

Level: Expert Specialists with years of training, often over hours of work
Scored: Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers: Agent task Many steps in a terminal, browser, desktop, app, game or robot.

Model scored in lab cards

System cards report it, from 1 lab

GPT-6 Astra leads with 90.9%

Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.

Bars Table

90.9 GPT-6 Astra

1 GPT-6 Astra 90.9%

#	Model	Score	Setting	Date	Source
1	GPT-6 Astra	90.9%		Sep 2026	Model card

The best score passed 85% in Sep 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

Reported by the lab in its model card

Labs report it only in system cards

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.

9 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

Date	Document	Score	As the document writes it	Section
3 Sep 2026	GPT-6 Astra system card	90.9	Irregular Atomic Challenges, 20 of 22 solved (10.1.2.6)	Cyber risk
9 Jul 2026	GPT-5.6 Sol system card		Irregular Atomic Challenges, all 22 medium and hard challenges solved at least once, average success Network Attack Simulation 98%, Vulnerability Research and Exploitation 91%, Evasion 56%, no overall score (9.1.2.5)	Cyber risk
25 Jun 2026	GPT-5.6 Sol system card		Irregular Atomic Challenges, all 22 medium and hard challenges solved at least once, average success Network Attack Simulation 98%, Vulnerability Research and Exploitation 91%, Evasion 56%, no overall score (9.1.2.5)	Cyber risk
23 Apr 2026	GPT-5.5 system card		Irregular atomic challenge suite, network attack simulation 98%, vulnerability research and exploitation 92%, evasion 54%, no overall score (9.1.2.5)	Cyber risk
5 Mar 2026	GPT-5.4 system card		Irregular atomic challenge suite, network attack simulation 88%, vulnerability research and exploitation 73%, evasion 48%, no overall score (5.1.2.4)	Cyber risk
5 Feb 2026	GPT-5.3 Codex system card		External Evaluations by Irregular, average success rates for Network Attack Simulation 86%, Vulnerability Research and Exploitation 72%, Evasion 53%, no overall score (5.1.2.4)	Cyber risk
18 Dec 2025	GPT-5.2 Codex system card addendum		External Evaluations by Irregular, medium and hard challenges only, average success rates Network Attack Simulation 79%, Vulnerability Research and Exploitation 80%, Evasion 49%, no overall score (5.1.2.4)	Cyber risk
11 Dec 2025	GPT-5.2 system card addendum		External Evaluations by Irregular, v1 atomic challenge suite, Vulnerability Research and Exploitation 83%, Network Attack Simulation 100%, Evasion 73%, no overall score (4.1.2.4)	Cyber risk
18 Nov 2025	GPT-5.1-Codex-Max system card		External evaluations by Irregular, average success rates Network Attack Simulation 37%, Vulnerability Discovery and Exploitation 41%, Evasion 43%, solved 17 of 18 easy, 9 of 17 medium and 0 of 6 hard challenges, no overall score (5.1.2.4)	Cyber risk

GPT-6 Astra leads with 90.9%

The best score passed 85% in Sep 2026

Labs report it only in system cards

9 system cards report it, from 1 lab

More in Cybersecurity