Benchmark Almanac

CyberBench (Vals)

GPT-5.6 Sol 88.1% leads since Jul 2026 Not in model cards

Agents work on OSS-Fuzz memory-safety bugs from ARVO images in two tracks. They write a proof-of-concept input that crashes the target, or patch the source so the crash stops and tests still pass.

The PoC track follows CyberGym's method. Runs used mini-SWE-agent. Not the 2024 CyberBench by Liu et al. Archived after v1.1 replaced it.

Made by Vals AI Released Jun 2026 Finding and exploiting real bugs Tasks kept private Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
Leaderboard vals.ai
26
Models scored on its own leaderboard

GPT-5.6 Sol leads with 88.1% among models released by Sep 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in Sep 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

26 models. Scroll sideways for the rest.

  1. 88.1 GPT-5.6 Sol
  2. 83.9 GPT-5.6 Luna
  3. 80.5 GPT-5.5
  4. 79.0 Kimi K3
  5. 78.8 DeepSeek V4.1 Flash
  6. 77.5 DeepSeek V4 Flash 0731
  7. 77.4 GLM-5.2
  8. 76.3 MiniMax-M3
  9. 74.3 Kimi K2.6
  10. 72.9 GPT-5.4
  11. 72.3 Gemini 3.5 Flash-Lite
  12. 71.5 GLM-5.3-Flash
  13. 69.3 Gemini 3.5 Flash
  14. 68.8 Inkling-Small
  15. 68.6 DeepSeek-V4-Pro
  16. 66.7 DeepSeek-V4-Flash
  17. 64.1 Inkling
  18. 57.5 Tencent Hy4 preview
  19. 52.1 Claude Opus 4.7
  20. 50.9 Claude Opus 4.8
  21. 48.5 Gemini 3.6 Flash
  22. 47.5 Grok 4.3
  23. 43.8 Gemini 3.8 Flash
  24. 40.7 Claude Opus 5
  25. 37.5 Qwen3.7-Plus
  26. 36.4 Gemini 3.1 Pro
  1. 1 GPT-5.6 Sol 88.1%
  2. 2 GPT-5.6 Luna 83.9%
  3. 3 GPT-5.5 80.5%
  4. 4 Kimi K3 79%
  5. 5 DeepSeek V4.1 Flash 78.8%
  6. 6 DeepSeek V4 Flash 0731 77.5%
  7. 7 GLM-5.2 77.4%
  8. 8 MiniMax-M3 76.3%
  9. 9 Kimi K2.6 74.3%
  10. 10 GPT-5.4 xhigh 72.9%
  11. 11 Gemini 3.5 Flash-Lite 72.3%
  12. 12 GLM-5.3-Flash 71.5%
  13. 13 Gemini 3.5 Flash 69.3%
  14. 14 Inkling-Small 68.8%
  15. 15 DeepSeek-V4-Pro 68.6%
  16. 16 DeepSeek-V4-Flash 66.7%
  17. 17 Inkling 64.1%
  18. 18 Tencent Hy4 preview 57.5%
  19. 19 Claude Opus 4.7 52.1%
  20. 20 Claude Opus 4.8 50.9%
  21. 21 Gemini 3.6 Flash 48.5%
  22. 22 Grok 4.3 47.5%
  23. 23 Gemini 3.8 Flash 43.8%
  24. 24 Claude Opus 5 40.7%
  25. 25 Qwen3.7-Plus 37.5%
  26. 26 Gemini 3.1 Pro 36.4%
#ModelScoreSettingDateSource
1 GPT-5.6 Sol 88.1% Jul 2026 Official leaderboard
2 GPT-5.6 Luna 83.9% Jul 2026 Official leaderboard
3 GPT-5.5 80.5% Apr 2026 Official leaderboard
4 Kimi K3 79% Jul 2026 Official leaderboard
5 DeepSeek V4.1 Flash 78.8% Sep 2026 Official leaderboard
6 DeepSeek V4 Flash 0731 77.5% Jul 2026 Official leaderboard
7 GLM-5.2 77.4% Jun 2026 Official leaderboard
8 MiniMax-M3 76.3% Jun 2026 Official leaderboard
9 Kimi K2.6 74.3% Apr 2026 Official leaderboard
10 GPT-5.4 72.9% xhigh Mar 2026 Official leaderboard
11 Gemini 3.5 Flash-Lite 72.3% Jul 2026 Official leaderboard
12 GLM-5.3-Flash 71.5% Aug 2026 Official leaderboard
13 Gemini 3.5 Flash 69.3% May 2026 Official leaderboard
14 Inkling-Small 68.8% Jul 2026 Official leaderboard
15 DeepSeek-V4-Pro 68.6% Apr 2026 Official leaderboard
16 DeepSeek-V4-Flash 66.7% Apr 2026 Official leaderboard
17 Inkling 64.1% Jul 2026 Official leaderboard
18 Tencent Hy4 preview 57.5% Aug 2026 Official leaderboard
19 Claude Opus 4.7 52.1% Apr 2026 Official leaderboard
20 Claude Opus 4.8 50.9% May 2026 Official leaderboard
21 Gemini 3.6 Flash 48.5% Jul 2026 Official leaderboard
22 Grok 4.3 47.5% Apr 2026 Official leaderboard
23 Gemini 3.8 Flash 43.8% Sep 2026 Official leaderboard
24 Claude Opus 5 40.7% Jul 2026 Official leaderboard
25 Qwen3.7-Plus 37.5% Jun 2026 Official leaderboard
26 Gemini 3.1 Pro 36.4% Feb 2026 Official leaderboard

The best score, 88.1%, costs $1.15 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI DeepSeek Cost frontier One model at each effort level
0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-5.6 Sol DeepSeek V4.1 Flash 0% 20% 40% 60% 80% 100% $0.01 $0.10 $1.00 $10 Cost per task, log scale GPT-5.6 Sol DeepSeek V4.1 Flash
ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5.6 Sol 88.1% $1.15
GPT-5.6 Luna 83.9%
GPT-5.5 80.5% $2.81
Kimi K3 79% $1.48
DeepSeek V4.1 Flash 78.8% $0.07
DeepSeek V4 Flash 0731 77.5% $0.08
GLM-5.2 77.4% $1.21
MiniMax-M3 76.3% $1.74
Kimi K2.6 74.3% $1.05
GPT-5.4 xhigh 72.9% $2.95
Gemini 3.5 Flash-Lite 72.3% $0.55
GLM-5.3-Flash 71.5% $0.11
Gemini 3.5 Flash 69.3% $2.00
Inkling-Small 68.8% $0.19
DeepSeek-V4-Pro 68.6% $0.68
DeepSeek-V4-Flash 66.7% $0.20
Inkling 64.1% $4.85
Tencent Hy4 preview 57.5% $0.49
Claude Opus 4.7 52.1% $4.09
Claude Opus 4.8 50.9% $1.71
Gemini 3.6 Flash 48.5% $0.87
Grok 4.3 47.5% $2.56
Gemini 3.8 Flash 43.8% $1.31
Claude Opus 5 40.7% $2.87
Qwen3.7-Plus 37.5% $0.10
Gemini 3.1 Pro 36.4% $2.10

2 models have held number one since its release in Jun 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Passed 85% in Jul 2026, by GPT-5.6 Sol
GPT-5.6 Sol 88.1%

The best score passed 85% in Jul 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.6 Sol GPT-5.5 Kimi K2.6 Gemini 3.1 Pro 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.6 Sol GPT-5.5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

OpenAI Meta
CyberBench v1.1 (Vals) 4 leaders
Muse Wonder 80.4%
CyberBench (Vals) Passed 85% in Jul 2026, by GPT-5.6 Sol
GPT-5.6 Sol 88.1%
CyberBench v1.1 (Vals) 2 leaders
Muse Wonder 80.4%
Replaced by CyberBench v1.1 (Vals)

More in Cybersecurity

All 111 in Cybersecurity
  • ExploitGym In use 869 real-vulnerability exploit-development tasks (userspace, V8, Linux kernel). Success means capturing a flag through the intended vulnerability. GPT-6 Astra 42.4%
  • ExploitBench In use 41 recent V8 JavaScript-engine N-day vulnerabilities. Progress measured by 16 grader-verified capability flags across five tiers. GPT-6 Astra 100%
  • CWE-bench In use Agents audit a repository and write a patch for a security flaw, without being told where the bug is. Covers six programming languages. Gemini 4 Carbon 71%
  • Real-world vulnerability discovery In use Agents search popular open-source codebases to rediscover about 1,200 recent confirmed vulnerabilities across many languages. Reports share discovered. Gemini 4 Argon 86%
  • Artificial Analysis Cyber Index In use Agents do enterprise cyber defense tasks from three benchmarks, CWE-Bench-AA, DeepsecBench-AA and CyberGym-E2E-AA, combined into one success rate. Mistral Large 4 (Le Chonk, open weights) 51%
  • Big Sleep Evaluation (Google) In use Agents search large real codebases such as Chrome and Safari for critical, hard to find vulnerabilities. Scored by pass@1 success rate, run without safety guardrails. Gemini 3.5 Flash Cyber 72%