Divergence attack (extractable memorization)
Gemini 2.5 Flash Jun 2025 59% Only in system cards Models are prompted to repeat one token many times. The test measures how often they diverge and emit memorized training text.
Google runs 3,750 prompts over 125 single-token characters, following Nasr et al. 2023.
In the authors’ words
This paper studies extractable memorization: training data that an adversary can efficiently extract by querying a machine learning model without prior knowledge of the training dataset. We show an adversary can extract gigabytes of training data from open-source language models like Pythia or GPT-Neo, semi-open models like LLaMA or Falcon, and closed models like ChatGPT. Existing techniques from the literature suffice to attack unaligned models; in order to attack the aligned ChatGPT, we develop a new divergence attack that causes the model to diverge from its chatbot-style generations and emit training data at a rate 150x higher than when behaving properly. Our methods show practical attacks can recover far more data than previously thought, and reveal that current alignment techniques do not eliminate memorization.
- Level
- Everyday Most adults, without training
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Open-ended Free-form text, documents, designs or media judged as a whole.
2 models share the top score of 59%
Best score per model as labs report them in their own cards. Labs pick their own settings, so these compare less cleanly than independent runs.
- 1 Gemini 2.5 Flash Jun 2025 59%
- 2 Gemini 2.5 Pro Jun 2025 59%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | Gemini 2.5 Flash Jun 2025 | 59% | Jul 2025 | Model card | |
| 2 | Gemini 2.5 Pro Jun 2025 | 59% | Jul 2025 | Model card |
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
Labs report it only in system cards
We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.
2 system cards report it, from 1 lab
Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.
| Date | Document | Score | As the document writes it | Section |
|---|---|---|---|---|
| 7 Jul 2025 | Gemini 2.5 Pro technical report | 59 | Divergence attack: divergence occurs roughly 59% of the time for the Gemini 2.5 family; about 0.2% of diverged text classified as memorization (Section 5.6) | Safety |
| 7 Jul 2025 | Gemini 2.5 Flash technical report | 59 | Divergence attack: divergence occurs roughly 59% of the time for the Gemini 2.5 family; about 0.2% of diverged text classified as memorization (Section 5.6) | Safety |
More in Safety and alignment
- FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
- StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is.
Inkling 98.6% - AgentDojo In use Agents do tool-use tasks while tools and data carry prompt injections. Attack success rate counts hijacked runs. Grok 4 Fast 0%
- MASK In use Measures whether a model contradicts its own elicited beliefs under pressure to lie. Honesty separated from accuracy.
Muse Spark Open 98.1% - Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. GPT-6 Astra 8.50%
- Agentic misalignment suite (Anthropic) In use Agents in simulated company settings face threats or goal conflicts that invite blackmail, leaking or sabotage. Score is the misaligned action rate. Claude Sonnet 4.5 0%