Benchmark Almanac

SlopCodeBench

GPT-5.5 28.1% leads since Apr 2026 Not in model cards

Agents repeatedly extend their own code across 36 problems and 196 checkpoints as the specification changes. Tests check correctness, and metrics track structural erosion and verbosity.

The paper finds that no agent fully solves any problem end to end.

In the authors’ words

Software development is iterative, yet agentic coding benchmarks hide design issues through their single-shot setup. Recent iterative benchmarks attempt to remedy this but heavily constrain an agent's design decision space, making it impossible to faithfully measure how their decisions shape future extensions. We introduce SlopCodeBench, a benchmark of 36 problems and 196 checkpoints where agents repeatedly extend their own solutions. Unlike prior iterative benchmarks, our evolving specifications demand architectural decisions but leave internal structure to the agent. We measure two forms of degradation: structural erosion (concentrated complexity) and verbosity (redundant code). Evaluating 15 coding agents across open and closed models, we find that no agent fully solves any problem end-to-end, and the best agent passes 14.8% of checkpoints. Quality degrades across checkpoints, with structural erosion rising in 77% of trajectories and verbosity in 75.5%. Compared to 473 open-source Python repositories, agent code is 2.3x more verbose and 2.0x more eroded, and the human repositories degrade less often and by smaller margins across their git histories. Explicit quality guidance reduces initial verbosity and erosion by up to a third, without affecting degradation rates. SlopCodeBench provides the first measurement of code degradation under iterative extension, revealing that agents pass checkpoints while producing code that erodes and bloats with each turn.

Abstract of SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks (arXiv:2603.24755, CC0)
Made by University of Wisconsin-Madison (SprocketLab) Released Mar 2026 Building whole projects All tasks public Last model added Apr 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
14
Models scored on its own leaderboard

GPT-5.5 leads with 28.1% among models released by Apr 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in Apr 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 1 GPT-5.5 high 28.1%
  2. 2 GPT-5.3 Codex high 26%
  3. 3 GPT-5.4 high 23.5%
  4. 4 GPT-5.2 Codex high 21.9%
  5. 5 Claude Opus 4.6 high 20.9%
  6. 6 Claude Opus 4.7 high 20.9%
  7. 7 Kimi K2.6 high 18.9%
  8. 8 Claude Opus 4.5 high 17.4%
  9. 9 Claude Sonnet 4.6 high 16.8%
  10. 10 GPT-5.4 Mini high 13.8%
  11. 11 GLM-5.1 high 13.8%
  12. 12 Kimi K2.5 high 9.69%
  13. 13 GPT-5.3-Codex-Spark high 8.16%
  14. 14 MiniMax-M2.7 high 4.08%
#ModelScoreSettingDateSource
1 GPT-5.5 28.1% high, Codex Apr 2026 Official leaderboard
2 GPT-5.3 Codex 26% high, Codex Feb 2026 Official leaderboard
3 GPT-5.4 23.5% high, Codex Mar 2026 Official leaderboard
4 GPT-5.2 Codex 21.9% high, Codex Dec 2025 Official leaderboard
5 Claude Opus 4.6 20.9% high, Claude Code Feb 2026 Official leaderboard
6 Claude Opus 4.7 20.9% high, Claude Code Apr 2026 Official leaderboard
7 Kimi K2.6 18.9% high, Kimi CLI Apr 2026 Official leaderboard
8 Claude Opus 4.5 17.4% high, Claude Code Nov 2025 Official leaderboard
9 Claude Sonnet 4.6 16.8% high, Claude Code Feb 2026 Official leaderboard
10 GPT-5.4 Mini 13.8% high, Codex Mar 2026 Official leaderboard
11 GLM-5.1 13.8% high, Claude Code Apr 2026 Official leaderboard
12 Kimi K2.5 9.69% high, Kimi CLI Jan 2026 Official leaderboard
13 GPT-5.3-Codex-Spark 8.16% high, Codex Feb 2026 Official leaderboard
14 MiniMax-M2.7 4.08% high, Claude Code Mar 2026 Official leaderboard

The best score, 28.1%, costs $1.51 a task

Runs on the cost frontier score higher than any run that's cheaper. GPT-5.3 Codex at high comes within 2 points of the best for $0.69 a task, about half the cost. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Moonshot AI OpenAI Cost frontier One model at each effort level
0% 10% 20% 30% $0.10 $0.20 $0.50 $1.00 $2.00 $5.00 $10 Cost per task, log scale GPT-5.5, high GPT-5.3 Codex, high GPT-5.4 Mini, high Kimi K2.5, high GPT-5.3-Codex-Spark, high 0% 10% 20% 30% $0.10 $1.00 $10 Cost per task, log scale GPT-5.5, high GPT-5.3 Codex, high
ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5.5 high Codex 28.1% $1.51
GPT-5.3 Codex high Codex 26% $0.69
GPT-5.4 high Codex 23.5% $0.82
GPT-5.2 Codex high Codex 21.9% $0.85
Claude Opus 4.6 high Claude Code 20.9% $3.17
Claude Opus 4.7 high Claude Code 20.9% $2.17
Kimi K2.6 high Kimi CLI 18.9% $0.74
Claude Opus 4.5 high Claude Code 17.4% $2.53
Claude Sonnet 4.6 high Claude Code 16.8% $1.96
GPT-5.4 Mini high Codex 13.8% $0.45
GLM-5.1 high Claude Code 13.8% $1.47
Kimi K2.5 high Kimi CLI 9.69% $0.33
GPT-5.3-Codex-Spark high Codex 8.16% $0.20
GLM-5.1 high OpenCode 8.16% $0.59
Kimi K2.5 high Claude Code 7.14% $1.07
MiniMax-M2.7 high Claude Code 4.08% $0.33

2 models have held number one since its release in Mar 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard Not tested since Apr 2026
GPT-5.5 28.1%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.5 GPT-5.3 Codex Claude Opus 4.5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) GPT-5.5 Claude Opus 4.5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Coding

All 243 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Opus 5.5 93.9%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471