Benchmark Almanac

CWE-bench 2

Claude Fable 5.2 75% leads since November 2026
Not in model cards

Agents audit and patch vulnerabilities in 250 codebases. The score is pass@1 under deterministic verification.

The board publishes pass@4 and average API cost per rollout. Runs use high reasoning and a one-hour task limit.

Made by
Collinear AI
Released
November 2026
Kind
Secure code and review
Level
Expert
Scored
Checked automatically
Answers
Code
Tasks
Public, plus a private test set
Strongest model tested
Claude Fable 5.2 #1 on the capability index
Last model added
November 2026
Scores refreshed
November 13th, 2026
14
Models scored on its own leaderboard

Claude Fable 5.2 leads with 75%

Best score per model from the benchmark's own leaderboard, read on November 13th, 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

14 models. Scroll sideways for the rest.

  1. 75.0 Claude Fable 5.2
  2. 68.1 Gemini 4 Hydrogen
  3. 67.1 Claude Opus 5.5
  4. 66.1 Muse Wonder
  5. 65.0 Gemini 4 Argon
  6. 64.2 GPT-6 Astra
  7. 64.1 Tencent Hy4
  8. 57.6 DeepSeek V4.1 Flash 1017
  9. 54.7 Grok 4.7
  10. 47.3 Tencent Hy4 preview
  11. 45.6 Mistral Large 4
  12. 45.4 Qwen3.8 Max (0902)
  13. 45.3 Kimi K3.1 Pro
  14. 42.0 GLM-5.3
  1. 1 Claude Fable 5.2 high reasoning; 1h limit 75%
  2. 2 Gemini 4 Hydrogen high reasoning; 1h limit 68.1%
  3. 3 Claude Opus 5.5 high reasoning; 1h limit 67.1%
  4. 4 Muse Wonder high reasoning; 1h limit 66.1%
  5. 5 Gemini 4 Argon high reasoning; 1h limit 65%
  6. 6 GPT-6 Astra high reasoning; 1h limit 64.2%
  7. 7 Tencent Hy4 high reasoning; 1h limit 64.1%
  8. 8 DeepSeek V4.1 Flash 1017 high reasoning; 1h limit 57.6%
  9. 9 Grok 4.7 high reasoning; 1h limit 54.7%
  10. 10 Tencent Hy4 preview high reasoning; 1h limit 47.3%
  11. 11 Mistral Large 4 high reasoning; 1h limit 45.6%
  12. 12 Qwen3.8 Max (0902) high reasoning; 1h limit 45.4%
  13. 13 Kimi K3.1 Pro high reasoning; 1h limit 45.3%
  14. 14 GLM-5.3 high reasoning; 1h limit 42%

Source: Official CWE-bench 2 leaderboard

#ModelScoreSettingDateSource
1 Claude Fable 5.2 75% high reasoning; 1h limit, Claude Code Nov 2026 Official leaderboard
2 Gemini 4 Hydrogen 68.1% high reasoning; 1h limit, Antigravity Nov 2026 Official leaderboard
3 Claude Opus 5.5 67.1% high reasoning; 1h limit, Claude Code Sep 2026 Official leaderboard
4 Muse Wonder 66.1% high reasoning; 1h limit, opencode Oct 2026 Official leaderboard
5 Gemini 4 Argon 65% high reasoning; 1h limit, Antigravity Sep 2026 Official leaderboard
6 GPT-6 Astra 64.2% high reasoning; 1h limit, Codex Sep 2026 Official leaderboard
7 Tencent Hy4 64.1% high reasoning; 1h limit, Claude Code Nov 2026 Official leaderboard
8 DeepSeek V4.1 Flash 1017 57.6% high reasoning; 1h limit, opencode Oct 2026 Official leaderboard
9 Grok 4.7 54.7% high reasoning; 1h limit, opencode Sep 2026 Official leaderboard
10 Tencent Hy4 preview 47.3% high reasoning; 1h limit, opencode Aug 2026 Official leaderboard
11 Mistral Large 4 45.6% high reasoning; 1h limit, opencode Oct 2026 Official leaderboard
12 Qwen3.8 Max (0902) 45.4% high reasoning; 1h limit, opencode Sep 2026 Official leaderboard
13 Kimi K3.1 Pro 45.3% high reasoning; 1h limit, opencode Oct 2026 Official leaderboard
14 GLM-5.3 42% high reasoning; 1h limit, opencode Aug 2026 Official leaderboard

Pass@1 and average cost per rollout

The line joins the runs that no other run beats on both measures. We redrew this chart from the numbers on the official leaderboard.

40% 50% 60% 70% 80% $0$2$4$6$8 Mean cost per rollout ($), lower is better Pass@1 GLM-5.3 (high reasoning; 1h limit): Pass@1 42%, mean cost per rollout $0.58 Tencent Hy4 preview (high reasoning; 1h limit): Pass@1 47.3%, mean cost per rollout $0.43 Qwen3.8 Max (0902) (high reasoning; 1h limit): Pass@1 45.4%, mean cost per rollout $2.06 GPT-6 Astra (high reasoning; 1h limit): Pass@1 64.2%, mean cost per rollout $3.87 Grok 4.7 (high reasoning; 1h limit): Pass@1 54.7%, mean cost per rollout $2.97 Gemini 4 Argon (high reasoning; 1h limit): Pass@1 65%, mean cost per rollout $3.7 Mistral Large 4 (high reasoning; 1h limit): Pass@1 45.6%, mean cost per rollout $0.37 Muse Wonder (high reasoning; 1h limit): Pass@1 66.1%, mean cost per rollout $2.73 Kimi K3.1 Pro (high reasoning; 1h limit): Pass@1 45.3%, mean cost per rollout $0.96 Claude Opus 5.5 (high reasoning; 1h limit): Pass@1 67.1%, mean cost per rollout $1.85 DeepSeek V4.1 Flash 1017 (high reasoning; 1h limit): Pass@1 57.6%, mean cost per rollout $0.04 Tencent Hy4 (high reasoning; 1h limit): Pass@1 64.1%, mean cost per rollout $1.35 Claude Fable 5.2 (high reasoning; 1h limit): Pass@1 75%, mean cost per rollout $7.85 Gemini 4 Hydrogen (high reasoning; 1h limit): Pass@1 68.1%, mean cost per rollout $2.63 Claude Fable 5.2 (high reasoning; 1h limit)Gemini 4 Hydrogen (high reasoning; 1h limit)Claude Opus 5.5 (high reasoning; 1h limit)Tencent Hy4 (high reasoning; 1h limit)DeepSeek V4.1 Flash 1017 (high reasoning; 1h limit) 40% 50% 60% 70% 80% $0$2$4$6$8 Mean cost per rollout ($), lower is better Pass@1 GLM-5.3 (high reasoning; 1h limit): Pass@1 42%, mean cost per rollout $0.58 Tencent Hy4 preview (high reasoning; 1h limit): Pass@1 47.3%, mean cost per rollout $0.43 Qwen3.8 Max (0902) (high reasoning; 1h limit): Pass@1 45.4%, mean cost per rollout $2.06 GPT-6 Astra (high reasoning; 1h limit): Pass@1 64.2%, mean cost per rollout $3.87 Grok 4.7 (high reasoning; 1h limit): Pass@1 54.7%, mean cost per rollout $2.97 Gemini 4 Argon (high reasoning; 1h limit): Pass@1 65%, mean cost per rollout $3.7 Mistral Large 4 (high reasoning; 1h limit): Pass@1 45.6%, mean cost per rollout $0.37 Muse Wonder (high reasoning; 1h limit): Pass@1 66.1%, mean cost per rollout $2.73 Kimi K3.1 Pro (high reasoning; 1h limit): Pass@1 45.3%, mean cost per rollout $0.96 Claude Opus 5.5 (high reasoning; 1h limit): Pass@1 67.1%, mean cost per rollout $1.85 DeepSeek V4.1 Flash 1017 (high reasoning; 1h limit): Pass@1 57.6%, mean cost per rollout $0.04 Tencent Hy4 (high reasoning; 1h limit): Pass@1 64.1%, mean cost per rollout $1.35 Claude Fable 5.2 (high reasoning; 1h limit): Pass@1 75%, mean cost per rollout $7.85 Gemini 4 Hydrogen (high reasoning; 1h limit): Pass@1 68.1%, mean cost per rollout $2.63 Claude Fable 5.2 (high reasoning; 1h limit)Gemini 4 Hydrogen (high reasoning; 1h limit)Claude Opus 5.5 (high reasoning; 1h limit)Tencent Hy4 (high reasoning; 1h limit)DeepSeek V4.1 Flash 1017 (high reasoning; 1h limit)
ModelSettingPass@1Mean cost per rollout
Claude Fable 5.2 high reasoning; 1h limit, Claude Code 75% $7.85
Gemini 4 Hydrogen high reasoning; 1h limit, Antigravity 68.1% $2.63
Claude Opus 5.5 high reasoning; 1h limit, Claude Code 67.1% $1.85
Muse Wonder high reasoning; 1h limit, opencode 66.1% $2.73
Gemini 4 Argon high reasoning; 1h limit, Antigravity 65% $3.7
GPT-6 Astra high reasoning; 1h limit, Codex 64.2% $3.87
Tencent Hy4 high reasoning; 1h limit, Claude Code 64.1% $1.35
DeepSeek V4.1 Flash 1017 high reasoning; 1h limit, opencode 57.6% $0.04
Grok 4.7 high reasoning; 1h limit, opencode 54.7% $2.97
Tencent Hy4 preview high reasoning; 1h limit, opencode 47.3% $0.43
Mistral Large 4 high reasoning; 1h limit, opencode 45.6% $0.37
Qwen3.8 Max (0902) high reasoning; 1h limit, opencode 45.4% $2.06
Kimi K3.1 Pro high reasoning; 1h limit, opencode 45.3% $0.96
GLM-5.3 high reasoning; 1h limit, opencode 42% $0.58

The best score, 75%, costs $7.85 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Anthropic DeepSeek Tencent Google DeepMind Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5.2, high reasoning; 1h limit Gemini 4 Hydrogen, high reasoning; 1h limit Tencent Hy4, high reasoning; 1h limit DeepSeek V4.1 Flash 1017, high reasoning; 1h limit 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale Claude Fable 5.2, high reasoning; 1h limit Gemini 4 Hydrogen, high reasoning; 1h limit

Source: Official CWE-bench 2 leaderboard

ModelEffortHarnessScoreCost per taskOutput tokens
Claude Fable 5.2 high reasoning; 1h limit Claude Code 75% $7.85
Gemini 4 Hydrogen high reasoning; 1h limit Antigravity 68.1% $2.63
Claude Opus 5.5 high reasoning; 1h limit Claude Code 67.1% $1.85
Muse Wonder high reasoning; 1h limit opencode 66.1% $2.73
Gemini 4 Argon high reasoning; 1h limit Antigravity 65% $3.70
GPT-6 Astra high reasoning; 1h limit Codex 64.2% $3.87
Tencent Hy4 high reasoning; 1h limit Claude Code 64.1% $1.35
DeepSeek V4.1 Flash 1017 high reasoning; 1h limit opencode 57.6% $0.04
Grok 4.7 high reasoning; 1h limit opencode 54.7% $2.97
Tencent Hy4 preview high reasoning; 1h limit opencode 47.3% $0.43
Mistral Large 4 high reasoning; 1h limit opencode 45.6% $0.37
Qwen3.8 Max (0902) high reasoning; 1h limit opencode 45.4% $2.06
Kimi K3.1 Pro high reasoning; 1h limit opencode 45.3% $0.96
GLM-5.3 high reasoning; 1h limit opencode 42% $0.58

2 models have held number one on the official leaderboard since its release in November 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 2 leaders
Claude Fable 5.2 75%

Source: Official CWE-bench 2 leaderboard

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) Claude Fable 5.2 Claude Opus 5.5 Tencent Hy4 preview GLM-5.3 0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) Claude Fable 5.2 Claude Opus 5.5

Source: Official CWE-bench 2 leaderboard

No lab model card reports it yet

We read the headline results of 522 launch posts from 52 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic OpenAI Google DeepMind
CWE-bench 2 5 leaders
Claude Fable 5.2 75%
CWE-bench 3 leaders
Gemini 4 Carbon 71%
CWE-bench 2 2 leaders
Claude Fable 5.2 75%

Source: Official leaderboards

Replaces CWE-bench

More in Cybersecurity

All 115 in Cybersecurity
  • ExploitGym In use 869 real-vulnerability exploit-development tasks (userspace, V8, Linux kernel). Success means capturing a flag through the intended vulnerability. GPT-6 Astra 42.4%
  • ExploitBench In use 41 recent V8 JavaScript-engine N-day vulnerabilities. Progress measured by 16 grader-verified capability flags across five tiers. GPT-6 Astra 100%
  • CyberBench v1.1 (Vals) In use Agents work on OSS-Fuzz memory-safety bugs from ARVO images in two tracks. They write a proof-of-concept input that crashes the target, or patch the source so the crash stops and tests still pass. Muse Wonder 80.4%
  • SRE-Bench In use Software reverse engineering without source: 262 binary instances from 19 private programs, each with six graded objectives. GPT-6 Astra 56.9%
  • CyScenarioBench In use Scenario-based long-horizon security challenges in sandboxes, organized in task, scenario and campaign tiers. Evaluation set kept private. Gemini 4 Argon 100%
  • ExploitBench - Internal Port In use Agents develop working exploits for recently disclosed vulnerabilities to limit contamination. Scored as arbitrary code execution success rate. GPT-6 Cyber 42%