SlopCodeBench
Agents repeatedly extend their own code across 36 problems and 196 checkpoints as the specification changes. Tests check correctness, and metrics track structural erosion and verbosity.
The paper finds that no agent fully solves any problem end to end.
In the authors’ words
Software development is iterative, yet agentic coding benchmarks hide design issues through their single-shot setup. Recent iterative benchmarks attempt to remedy this but heavily constrain an agent's design decision space, making it impossible to faithfully measure how their decisions shape future extensions. We introduce SlopCodeBench, a benchmark of 36 problems and 196 checkpoints where agents repeatedly extend their own solutions. Unlike prior iterative benchmarks, our evolving specifications demand architectural decisions but leave internal structure to the agent. We measure two forms of degradation: structural erosion (concentrated complexity) and verbosity (redundant code). Evaluating 15 coding agents across open and closed models, we find that no agent fully solves any problem end-to-end, and the best agent passes 14.8% of checkpoints. Quality degrades across checkpoints, with structural erosion rising in 77% of trajectories and verbosity in 75.5%. Compared to 473 open-source Python repositories, agent code is 2.3x more verbose and 2.0x more eroded, and the human repositories degrade less often and by smaller margins across their git histories. Explicit quality guidance reduces initial verbosity and erosion by up to a third, without affecting degradation rates. SlopCodeBench provides the first measurement of code degradation under iterative extension, revealing that agents pass checkpoints while producing code that erodes and bloats with each turn.
- Level
- Expert Specialists with years of training, often over hours of work
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Agent task Many steps in a terminal, browser, desktop, app, game or robot.
GPT-5.5 leads with 28.1% among models released by Apr 2026
Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.
The newest model with a score here came out in Apr 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- GPT-6 Astra
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- and 15 more
- 1 GPT-5.5 high 28.1%
- 2 GPT-5.3 Codex high 26%
- 3 GPT-5.4 high 23.5%
- 4 GPT-5.2 Codex high 21.9%
- 5 Claude Opus 4.6 high 20.9%
- 6 Claude Opus 4.7 high 20.9%
- 7 Kimi K2.6 high 18.9%
- 8 Claude Opus 4.5 high 17.4%
- 9 Claude Sonnet 4.6 high 16.8%
- 10 GPT-5.4 Mini high 13.8%
- 11 GLM-5.1 high 13.8%
- 12 Kimi K2.5 high 9.69%
- 13 GPT-5.3-Codex-Spark high 8.16%
- 14
MiniMax-M2.7 high 4.08%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-5.5 | 28.1% | high, Codex | Apr 2026 | Official leaderboard |
| 2 | GPT-5.3 Codex | 26% | high, Codex | Feb 2026 | Official leaderboard |
| 3 | GPT-5.4 | 23.5% | high, Codex | Mar 2026 | Official leaderboard |
| 4 | GPT-5.2 Codex | 21.9% | high, Codex | Dec 2025 | Official leaderboard |
| 5 | Claude Opus 4.6 | 20.9% | high, Claude Code | Feb 2026 | Official leaderboard |
| 6 | Claude Opus 4.7 | 20.9% | high, Claude Code | Apr 2026 | Official leaderboard |
| 7 | Kimi K2.6 | 18.9% | high, Kimi CLI | Apr 2026 | Official leaderboard |
| 8 | Claude Opus 4.5 | 17.4% | high, Claude Code | Nov 2025 | Official leaderboard |
| 9 | Claude Sonnet 4.6 | 16.8% | high, Claude Code | Feb 2026 | Official leaderboard |
| 10 | GPT-5.4 Mini | 13.8% | high, Codex | Mar 2026 | Official leaderboard |
| 11 | GLM-5.1 | 13.8% | high, Claude Code | Apr 2026 | Official leaderboard |
| 12 | Kimi K2.5 | 9.69% | high, Kimi CLI | Jan 2026 | Official leaderboard |
| 13 | GPT-5.3-Codex-Spark | 8.16% | high, Codex | Feb 2026 | Official leaderboard |
| 14 | | 4.08% | high, Claude Code | Mar 2026 | Official leaderboard |
The best score, 28.1%, costs $1.51 a task
Runs on the cost frontier score higher than any run that's cheaper. GPT-5.3 Codex at high comes within 2 points of the best for $0.69 a task, about half the cost. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-5.5 | high | Codex | 28.1% | $1.51 | |
| GPT-5.3 Codex | high | Codex | 26% | $0.69 | |
| GPT-5.4 | high | Codex | 23.5% | $0.82 | |
| GPT-5.2 Codex | high | Codex | 21.9% | $0.85 | |
| Claude Opus 4.6 | high | Claude Code | 20.9% | $3.17 | |
| Claude Opus 4.7 | high | Claude Code | 20.9% | $2.17 | |
| Kimi K2.6 | high | Kimi CLI | 18.9% | $0.74 | |
| Claude Opus 4.5 | high | Claude Code | 17.4% | $2.53 | |
| Claude Sonnet 4.6 | high | Claude Code | 16.8% | $1.96 | |
| GPT-5.4 Mini | high | Codex | 13.8% | $0.45 | |
| GLM-5.1 | high | Claude Code | 13.8% | $1.47 | |
| Kimi K2.5 | high | Kimi CLI | 9.69% | $0.33 | |
| GPT-5.3-Codex-Spark | high | Codex | 8.16% | $0.20 | |
| GLM-5.1 | high | OpenCode | 8.16% | $0.59 | |
| Kimi K2.5 | high | Claude Code | 7.14% | $1.07 | |
| | high | Claude Code | 4.08% | $0.33 |
2 models have held number one since its release in Mar 2026
Lab claims count from the day the card came out, independent runs from the model’s release.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
No lab model card reports it yet
We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.
More in Coding
- SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
- SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Opus 5.5 93.9%
- LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests.
DeepSeek-V4-Pro 93.5%
- DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
- SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases.
Seed2.1 Pro 59.8%
- Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance.
DeepSeek V4.1 Flash 3,471