Benchmark Almanac

Dictatorship Eval

Claude Fable 5 91.3% leads since Jun 2026 Not in model cards

Scenarios at six difficulty levels testing whether models help with authoritarian requests, from blunt asks to requests hidden in code edits. LLM judge, no self-grading.

Headline: all tested models complied when the request was disguised as an innocuous code edit. Author says the scenario library is open-sourced. No official repo found. Third-party extension: github.com/RandallSPQR/Dict_Eval_Hall.

Made by Free Systems (Andy Hall, Stanford Hoover) Released Apr 2026 Refusing harmful requests All tasks public Last model added Jul 2026 Scores refreshed 2 Nov 2026
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Open-ended Free-form text, documents, designs or media judged as a whole.
Paper Leaderboard
20
Models scored on its own leaderboard

2 models share the top score of 91.3% among models released by Jul 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in Jul 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

20 models. Scroll sideways for the rest.

  1. 91.3 Claude Fable 5
  2. 91.3 Muse Spark 1.1
  3. 90.3 Kimi K3
  4. 88.3 Claude Opus 4.7
  5. 86.4 Claude Opus 4.8
  6. 85.4 GPT-5.6 Sol
  7. 83.5 Claude Opus 4.6
  8. 83.5 Claude Sonnet 5
  9. 82.5 GPT-5.5
  10. 75.7 GPT-5.4
  11. 66.0 Grok 4.5
  12. 53.4 Gemini 3.1 Pro
  13. 46.6 GLM-5.2
  14. 45.6 Qwen3.7-Plus
  15. 38.8 Qwen 3.6 Plus
  16. 37.9 Grok 4.3
  17. 25.2 Grok 4.20
  18. 11.7 DeepSeek-V4-Pro
  19. 0.97 DeepSeek-V3.2
  20. 0.0 Llama 4 Maverick
  1. 1 Claude Fable 5 91.3%
  2. 2 Muse Spark 1.1 91.3%
  3. 3 Kimi K3 90.3%
  4. 4 Claude Opus 4.7 88.3%
  5. 5 Claude Opus 4.8 86.4%
  6. 6 GPT-5.6 Sol 85.4%
  7. 7 Claude Opus 4.6 83.5%
  8. 8 Claude Sonnet 5 83.5%
  9. 9 GPT-5.5 82.5%
  10. 10 GPT-5.4 75.7%
  11. 11 Grok 4.5 66%
  12. 12 Gemini 3.1 Pro 53.4%
  13. 13 GLM-5.2 46.6%
  14. 14 Qwen3.7-Plus 45.6%
  15. 15 Qwen 3.6 Plus 38.8%
  16. 16 Grok 4.3 37.9%
  17. 17 Grok 4.20 25.2%
  18. 18 DeepSeek-V4-Pro 11.7%
  19. 19 DeepSeek-V3.2 0.97%
  20. 20 Llama 4 Maverick 0%
#ModelScoreSettingDateSource
1 Claude Fable 5 91.3% Jun 2026 Official leaderboard
2 Muse Spark 1.1 91.3% Jul 2026 Official leaderboard
3 Kimi K3 90.3% Jul 2026 Official leaderboard
4 Claude Opus 4.7 88.3% Apr 2026 Official leaderboard
5 Claude Opus 4.8 86.4% May 2026 Official leaderboard
6 GPT-5.6 Sol 85.4% Jul 2026 Official leaderboard
7 Claude Opus 4.6 83.5% Feb 2026 Official leaderboard
8 Claude Sonnet 5 83.5% Jun 2026 Official leaderboard
9 GPT-5.5 82.5% Apr 2026 Official leaderboard
10 GPT-5.4 75.7% Mar 2026 Official leaderboard
11 Grok 4.5 66% Jul 2026 Official leaderboard
12 Gemini 3.1 Pro 53.4% Feb 2026 Official leaderboard
13 GLM-5.2 46.6% Jun 2026 Official leaderboard
14 Qwen3.7-Plus 45.6% Jun 2026 Official leaderboard
15 Qwen 3.6 Plus 38.8% Mar 2026 Official leaderboard
16 Grok 4.3 37.9% Apr 2026 Official leaderboard
17 Grok 4.20 25.2% Feb 2026 Official leaderboard
18 DeepSeek-V4-Pro 11.7% Apr 2026 Official leaderboard
19 DeepSeek-V3.2 0.97% Dec 2025 Official leaderboard
20 Llama 4 Maverick 0% Apr 2025 Official leaderboard

3 models have held number one since its release in Apr 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard Passed 85% in Apr 2026, by Claude Opus 4.7
Claude Fable 5 91.3%

The best score passed 85% in Apr 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Fable 5 Claude Opus 4.6 DeepSeek-V3.2 Llama 4 Maverick 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Fable 5 Claude Opus 4.6

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Safety and alignment

All 444 in Safety and alignment
  • FORTRESS In use Paired adversarial and benign national-security misuse prompts. Rubric-graded risk score on adversarial and over-refusal score on benign prompts. Claude Opus 5.5 5.78%
  • StrongREJECT In use Forbidden-prompt jailbreak benchmark with a rubric grader that scores how useful a non-refusal actually is. Inkling 98.6%
  • AgentDojo In use Agents do tool-use tasks while tools and data carry prompt injections. Attack success rate counts hijacked runs. Grok 4 Fast 0%
  • MASK In use Measures whether a model contradicts its own elicited beliefs under pressure to lie. Honesty separated from accuracy. Muse Spark Open 98.1%
  • Gray Swan IPI Arena benchmark In use Indirect prompt-injection attacks from public red-teaming competitions replayed on tool, coding and browser agents. Attack success rate at k attempts. GPT-6 Astra 8.50%
  • Agentic misalignment suite (Anthropic) In use Agents in simulated company settings face threats or goal conflicts that invite blackmail, leaking or sabotage. Score is the misaligned action rate. Claude Sonnet 4.5 0%