Anthropic automated behavioral audit
Claude Haiku 5.5 3.19 leads since Oct 2026 Only in system cards
Auditor agents run many simulated scenarios and score misuse cooperation, deception, sycophancy, self-preservation and other traits.
Petri is the related open-source tool, but the card does not name it.
Made by Anthropic Deception and alignment Tasks kept private Last in a system card Oct 2026
Last model added Oct 2026
- Scored
- Graded by a model A language model scores each response, usually against a rubric.
- Answers
- Agent task Many steps in a terminal, browser, desktop, app, game or robot.
6
Models scored in lab cards
18
System cards report it, from 1 lab
Claude Haiku 5.5 leads with 3.19
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
- 1 Claude Haiku 5.5 3.19
- 2 Claude Sonnet 5.5 2.87
- 3 Claude Sonnet 5 2.53
- 4 Claude Opus 5.5 2.49
- 5 Claude Opus 5 2.30
- 6 Claude Fable 5 2.06
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Claude Haiku 5.5 | 3.19 | Oct 2026 | Model card | |
| 2 | Claude Sonnet 5.5 | 2.87 | Sep 2026 | Model card | |
| 3 | Claude Sonnet 5 | 2.53 | Jun 2026 | Model card | |
| 4 | Claude Opus 5.5 | 2.49 | Sep 2026 | Model card | |
| 5 | Claude Opus 5 | 2.30 | Jul 2026 | Model card | |
| 6 | Claude Fable 5 | 2.06 | Jun 2026 | Model card |
4 models have held number one since 2023
Lab claims count from the day the card came out, independent runs from the model’s release.
Anthropic
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Reported by the lab in its model card
Labs report it only in system cards
We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.
18 system cards report it, from 1 lab
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 7 Oct 2026 | Claude Haiku 5.5 system card | 3.19 | Automated behavioral audit, overall misaligned behavior score on a 1 to 10 scale, lower is better. Figures 6.2.1.A to 6.2.7.A print scores for about 50 more dimensions (Figure 6.2.1.A) | Alignment |
| 28 Sep 2026 | Claude Sonnet 5.5 system card | 2.87 | Automated behavioral audit, overall misaligned behavior score on a 1 to 10 scale, lower is better, several dozen other dimensions charted (6.2.1, Figure 6.2.1.B) | Alignment |
| 22 Sep 2026 | Claude Opus 5.5 system card | 2.49 | Automated behavioral audit, overall misaligned behavior score on a 1 to 10 scale, lower is better, about 4,000 investigations. Figures 6.4.1.A to 6.4.7.A print scores for several dozen other dimensions (6.4.1, Figure 6.4.1.A) | Alignment |
| 1 Sep 2026 | Claude Fable 5.1 system card | Automated behavioral audit with Trust and Safety fallbacks enabled (Fable 5.1 falling back to Opus 4.8 and Opus 5), about 720 investigations, 1 to 10 scale, lower is better: biological weapons uplift 1.01, chemical weapons uplift 1.02, disallowed cyberoffense 1.13, cooperation with human misuse 1.78, targeted violence assistance 1.13, explosive weapons uplift 1.03; no overall score printed; main audit figures cover Mythos 5.1 only (Figure 6.4.9.A) | Alignment | |
| 24 Jul 2026 | Claude Opus 5 system card | 2.3 | Automated behavioral audit, overall misaligned behavior score on a 1 to 10 scale, lower is better, about 3,200 investigations, Mythos 5 judge, printed on chart. Misaligned with the constitution 2.00, cooperation with human misuse 1.56. Figures 6.4.1.A to 6.4.7.A hold many more dimensions (6.4.1, Figure 6.4.1.A) | Alignment |
| 30 Jun 2026 | Claude Sonnet 5 system card | 2.53 | Automated behavioral audit, overall misaligned behavior score on a 1 to 10 scale, lower is better, about 2,900 investigations (misaligned with the constitution 2.14). Figures 6.4.1.A to 6.4.6.A print about 40 more dimensions. Verbalized evaluation awareness in about 6% of transcripts (6.4.1, Figure 6.4.1.A) | Alignment |
| 9 Jun 2026 | Claude Fable 5 system card | 2.06 | Safeguards-on automated behavioral audit, Fable 5 behind production misuse classifiers with multi-model fallback, misuse-elicitation subset, about 440 investigations, helpful-only Opus 4.8 investigator limited to 50 actions, overall misaligned behavior score on a 1 to 10 scale, lower is better; cooperation with human misuse 1.58, harmful system prompt compliance 1.49, full-turn prefill susceptibility 1.56, controlled-substance synthesis 2.31, accepting unverifiable authorization 1.62; classifier block rate 43.5% per turn, 53.0% per branch, 77.7% per investigation, printed on charts (Figures 6.2.3.2.A and 6.2.3.2.B) | Alignment |
| 28 May 2026 | Claude Opus 4.8 system card | Automated behavioral audit, about 2,600 investigations with two investigator models, 1 to 10 scale, lower is better. Misaligned behavior and several dozen other dimensions, including undermining liberal democracy, charted without printed values (6.2.3.1, Figures 6.2.3.1.1.A to 6.2.3.1.6.A) | Alignment | |
| 16 Apr 2026 | Claude Opus 4.7 system card | Automated behavioral audit, about 2,300 investigations, several dozen metrics on a 1 to 10 scale including misaligned behavior, charts without printed values. Text says clear verbalized evaluation awareness in just under 1% of transcripts (6.2.3, Figure 6.2.3.2.A) | Alignment | |
| 7 Apr 2026 | Claude Mythos Preview system card | Automated behavioral audit, about 2,300 investigations per model with Opus 4.6 and Mythos Preview as investigators, several dozen dimensions on a 1 to 10 scale including misaligned behavior, charts without printed values (4.2.3.1, Figure 4.2.3.1.A) | Alignment | |
| 17 Feb 2026 | Claude Sonnet 4.6 system card | Automated behavioral audit, about 30 metrics on a 1 to 10 scale including misaligned behavior, charts without printed values (Figure 4.5.1.A) | Alignment | |
| 5 Feb 2026 | Claude Opus 4.6 system card | Automated behavioral audit, about 2,400 investigations per model, overall misaligned behavior and verbalized evaluation awareness plus about 30 further metrics on a 1 to 10 scale, charts without printed values (Figures 6.2.5.1.A and 6.2.5.2.A) | Alignment | |
| 24 Nov 2025 | Claude Opus 4.5 system card | Automated behavioral audit, about 1,800 investigations, overall misaligned behavior and evaluation awareness plus over 20 metrics, chart without printed values (Figures 6.2.A, 6.2.1.A). Text: misaligned behavior over 60% lower than Claude Opus 4.1, test speculation in fewer than 10% of investigations | Alignment | |
| 15 Oct 2025 | Claude Haiku 4.5 system card | Automated behavioral audit, 830 transcripts, 18 metrics including misaligned behavior, charts without printed values; text says 9% of transcripts show verbalized evaluation awareness (Figure 4.1.1.A) | Alignment | |
| 29 Sep 2025 | Claude Sonnet 4.5 system card | Automated behavioral audit, about 400 seed instructions, over 600 investigations per model, graded by Claude Opus 4.1 on 18 dimensions including misaligned behavior, charts without printed values. Text says misaligned behavior improved by more than 50% over Claude Sonnet 4 (7.1.3.1, Figure 7.1.3.1.A) | Alignment | |
| 5 Aug 2025 | Claude Opus 4.1 system card addendum | Automated behavioral audit, 1,160 transcripts from 290 seed instructions, 8 traits (concerning, cooperation with human misuse, brazen cooperation, initiative, deception toward user, audit situational awareness, whistleblowing, self preservation), chart without printed values. Text gives about 25% fewer misuse-cooperation transcripts than Claude Opus 4 (Figure 4.1.A) | Alignment | |
| 22 May 2025 | Claude Sonnet 4 system card | Automated behavioral audit, 414 conversations, 10 judge dimensions (concerning, no mitigating factors, initiative, persona, deceptive, situational awareness, self preservation, manipulative, whistleblowing, bizarre), chart without printed values (Fig. 4.2.1.A). Classifiers on the same transcripts: 18 situational awareness examples, 36 high-impact initiative cases (4.1.2.2, 4.1.9) | Alignment | |
| 22 May 2025 | Claude Opus 4 system card | Automated behavioral audit, 414 conversations, 10 judge dimensions (concerning, no mitigating factors, initiative, persona, deceptive, situational awareness, self preservation, manipulative, whistleblowing, bizarre), chart without printed values (Fig. 4.2.1.A). Classifiers on the same transcripts: 5/414 situational awareness, 58 high-impact initiative cases (4.1.2.2, 4.1.9) | Alignment |
More in Safety and alignment
- FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
- StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is.
Inkling 98.6% - AgentDojo In use Agents do tool-use tasks while tools and data carry prompt injections. Attack success rate counts hijacked runs. Grok 4 Fast 0%
- MASK In use Measures whether a model contradicts its own elicited beliefs under pressure to lie. Honesty separated from accuracy.
Muse Spark Open 98.1% - Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. GPT-6 Astra 8.50%
- Agentic misalignment suite (Anthropic) In use Agents in simulated company settings face threats or goal conflicts that invite blackmail, leaking or sabotage. Score is the misaligned action rate. Claude Sonnet 4.5 0%