Benchmark Almanac

Brokk Power Ranking

Claude Opus 5.5 37.9 leads since Sep 2026 Not in model cards

Models solve 181 real tickets from a 1.1 million line Rust codebase. Model judges compare solutions in pairs, and a Bradley-Terry fit gives each model a strength score.

Also reports speed and value per model. Updated 2026-10-29. Release month is the earliest Reddit post found, so check it.

Made by Brokk (SlopCop) Released Aug 2025 Fixing real repositories Tasks kept private Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Code A program or patch run against tests.
Leaderboard brokk.ai
20
Entries on its own leaderboard, 19 models we track

Claude Opus 5.5 leads with 37.9

Best score per entry from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart. Names in gray aren't in our model list: fine-tunes, agent systems, products and older models. They don't count toward leaders.

20 entries. Scroll sideways for the rest.

  1. 37.9 Claude Opus 5.5
  2. 29.6 GPT-6.1 Terra
  3. 26.5 Claude Sonnet 5.5
  4. 19.9 gpt-6-astra-mj
  5. 19.9 GPT-6.1 Sol
  6. 18.3 Claude Haiku 5.5
  7. 14.5 GPT-6 Astra
  8. 10.7 GPT-6 Luna
  9. 9.80 Claude Opus 5
  10. 8.20 GPT-6 Sol
  11. 8.10 GPT-5.6 Luna
  12. 7.70 Kimi K3
  13. 7.30 DeepSeek V4.1 Flash
  14. 7.10 GPT-5.6 Sol
  15. 5.00 Grok 4.7
  16. 4.20 Muse Spark 1.3
  17. 3.30 Kimi K3.1 Pro
  18. 2.80 GLM-5.3
  19. 1.60 GLM-5.3-Flash
  20. 1.40 Grok 4.6
  1. 1 Claude Opus 5.5 xhigh 37.9
  2. 2 GPT-6.1 Terra max 29.6
  3. 3 Claude Sonnet 5.5 xhigh 26.5
  4. 4 gpt-6-astra-mj xhigh 19.9
  5. 5 GPT-6.1 Sol high 19.9
  6. 6 Claude Haiku 5.5 max 18.3
  7. 7 GPT-6 Astra low 14.5
  8. 8 GPT-6 Luna max 10.7
  9. 9 Claude Opus 5 xhigh 9.80
  10. 10 GPT-6 Sol xhigh 8.20
  11. 11 GPT-5.6 Luna max 8.10
  12. 12 Kimi K3 max 7.70
  13. 13 DeepSeek V4.1 Flash max 7.30
  14. 14 GPT-5.6 Sol xhigh 7.10
  15. 15 Grok 4.7 high 5.00
  16. 16 Muse Spark 1.3 ultra 4.20
  17. 17 Kimi K3.1 Pro max 3.30
  18. 18 GLM-5.3 max 2.80
  19. 19 GLM-5.3-Flash max 1.60
  20. 20 Grok 4.6 xhigh 1.40
#EntryScoreSettingDateSource
1 Claude Opus 5.5 37.9 xhigh Sep 2026 Official leaderboard
2 GPT-6.1 Terra 29.6 max Oct 2026 Official leaderboard
3 Claude Sonnet 5.5 26.5 xhigh Sep 2026 Official leaderboard
4 gpt-6-astra-mj 19.9 xhigh Official leaderboard
5 GPT-6.1 Sol 19.9 high Sep 2026 Official leaderboard
6 Claude Haiku 5.5 18.3 max Oct 2026 Official leaderboard
7 GPT-6 Astra 14.5 low Sep 2026 Official leaderboard
8 GPT-6 Luna 10.7 max Sep 2026 Official leaderboard
9 Claude Opus 5 9.80 xhigh Jul 2026 Official leaderboard
10 GPT-6 Sol 8.20 xhigh Sep 2026 Official leaderboard
11 GPT-5.6 Luna 8.10 max Jul 2026 Official leaderboard
12 Kimi K3 7.70 max Jul 2026 Official leaderboard
13 DeepSeek V4.1 Flash 7.30 max Sep 2026 Official leaderboard
14 GPT-5.6 Sol 7.10 xhigh Jul 2026 Official leaderboard
15 Grok 4.7 5.00 high Sep 2026 Official leaderboard
16 Muse Spark 1.3 4.20 ultra Sep 2026 Official leaderboard
17 Kimi K3.1 Pro 3.30 max Oct 2026 Official leaderboard
18 GLM-5.3 2.80 max Aug 2026 Official leaderboard
19 GLM-5.3-Flash 1.60 max Aug 2026 Official leaderboard
20 Grok 4.6 1.40 xhigh Aug 2026 Official leaderboard

More effort adds a median 2 score

Models the leaderboard ran at two or more effort levels. We don’t have enough cost data here to say what the extra effort costs.

Lowest effort to highest, light to dark
Claude Opus 5.5
medium high xhigh
37.9
GPT-6.1 Terra
high xhigh max
29.6
Claude Sonnet 5.5
medium high xhigh
26.5
GPT-6.1 Sol
medium high xhigh
19.9
Claude Haiku 5.5
high xhigh max
18.3
GPT-6 Astra
low
14.5
GPT-6 Luna
xhigh
10.7
DeepSeek V4.1 Flash
high
7.30
Grok 4.7
high
5.00
Kimi K3.1 Pro
high
3.30
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Opus 5.5 xhigh 37.9
GPT-6.1 Terra max 29.6
Claude Opus 5.5 high 27.4
Claude Sonnet 5.5 xhigh 26.5
GPT-6.1 Sol high 19.9
Claude Haiku 5.5 max 18.3
Claude Sonnet 5.5 high 18.2
GPT-6.1 Sol medium 16.9
GPT-6.1 Sol xhigh 16.9
Claude Opus 5.5 medium 15.3
GPT-6 Astra low 14.5
Claude Haiku 5.5 xhigh 13.9
GPT-6 Astra medium 13.7
GPT-6.1 Terra xhigh 13.7
GPT-6 Luna max 10.7
Claude Opus 5 xhigh 9.80
GPT-6.1 Terra high 9.00
GPT-6 Luna xhigh 8.50
GPT-6 Sol xhigh 8.20
GPT-5.6 Luna max 8.10
Claude Haiku 5.5 high 7.90
Kimi K3 max 7.70
DeepSeek V4.1 Flash max 7.30
GPT-5.6 Sol xhigh 7.10
Claude Sonnet 5.5 medium 5.60
Grok 4.7 high 5.00
DeepSeek V4.1 Flash high 4.90
Grok 4.7 xhigh 4.60
Muse Spark 1.3 ultra 4.20
Kimi K3.1 Pro max 3.30
GLM-5.3 max 2.80
Kimi K3.1 Pro high 1.80
GLM-5.3-Flash max 1.60
Grok 4.6 xhigh 1.40

4 models have held number one since its release in Aug 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Anthropic
Own leaderboard 4 leaders
Claude Opus 5.5 37.9

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
0.00 10.0 20.0 30.0 40.0 2027 0.00 10.0 20.0 30.0 40.0 2027

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Coding

All 243 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Opus 5.5 93.9%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471