Benchmark Almanac

GSO-Bench

Claude Fable 5.1 88.2% leads since Sep 2026 Not in model cards

Agents speed up code in 102 tasks across 10 real codebases, given a performance test. Scored by the share matching the expert commit's speedup.

In the authors’ words

Developing high-performance software is a complex task that requires specialized expertise. We introduce GSO, a benchmark for evaluating language models' capabilities in developing high-performance software. We develop an automated pipeline that generates and executes performance tests to analyze repository commit histories to identify 102 challenging optimization tasks across 10 codebases, spanning diverse domains and programming languages. An agent is provided with a codebase and performance test as a precise specification, and tasked to improve the runtime efficiency, which is measured against the expert developer optimization. Our quantitative evaluation reveals that leading SWE-Agents struggle significantly, achieving less than 5% success rate, with limited improvements even with inference-time scaling. Our qualitative analysis identifies key failure modes, including difficulties with low-level languages, practicing lazy optimization strategies, and challenges in accurately localizing bottlenecks. We release the code and artifacts of our benchmark along with agent trajectories to enable future research.

Abstract of GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents (arXiv:2505.23671, CC0)
Made by GSO team (UC Berkeley et al.) Released May 2025 Speed and optimization All tasks public Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Code A program or patch run against tests.
gso-bench/gso 94 stars Paper Dataset Leaderboard gso-bench.github.io
30
Models scored on its own leaderboard

Claude Fable 5.1 leads with 88.2% among models released by Sep 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in Sep 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

30 models. Scroll sideways for the rest.

  1. 88.2 Claude Fable 5.1
  2. 79.4 GPT-6 Astra
  3. 78.4 Claude Fable 5
  4. 76.5 GPT-5.6 Sol
  5. 47.1 Claude Opus 4.8
  6. 44.1 Claude Opus 4.7
  7. 41.2 Claude Opus 4.6
  8. 40.2 GPT-5.5
  9. 37.3 Claude Sonnet 5
  10. 31.4 GPT-5.4
  11. 27.4 GPT-5.2
  12. 26.5 Claude Opus 4.5
  13. 22.6 Gemini 3.1 Pro
  14. 18.6 Gemini 3 Pro
  15. 14.7 Claude Sonnet 4.5
  16. 13.7 GPT-5.1
  17. 9.80 Gemini 3 Flash
  18. 8.82 o3
  19. 6.86 Claude Opus 4
  20. 6.86 GPT-5
  21. 4.90 Claude Sonnet 4
  22. 4.90 Kimi K2
  23. 4.90 Qwen3-Coder-480B-A35B
  24. 4.60 Claude 3.5 Sonnet (October 2024)
  25. 3.92 Gemini 2.5 Pro
  26. 3.80 Claude 3.7 Sonnet
  27. 3.60 o4-mini
  28. 2.94 GLM-4.5-Air
  29. 1.30 o3-mini
  30. 0.0 GPT-4o
  1. 1 Claude Fable 5.1 xhigh 88.2%
  2. 2 GPT-6 Astra xhigh 79.4%
  3. 3 Claude Fable 5 xhigh 78.4%
  4. 4 GPT-5.6 Sol xhigh 76.5%
  5. 5 Claude Opus 4.8 xhigh 47.1%
  6. 6 Claude Opus 4.7 high 44.1%
  7. 7 Claude Opus 4.6 high 41.2%
  8. 8 GPT-5.5 xhigh 40.2%
  9. 9 Claude Sonnet 5 xhigh 37.3%
  10. 10 GPT-5.4 xhigh 31.4%
  11. 11 GPT-5.2 high 27.4%
  12. 12 Claude Opus 4.5 26.5%
  13. 13 Gemini 3.1 Pro 22.6%
  14. 14 Gemini 3 Pro 18.6%
  15. 15 Claude Sonnet 4.5 14.7%
  16. 16 GPT-5.1 high 13.7%
  17. 17 Gemini 3 Flash 9.80%
  18. 18 o3 high 8.82%
  19. 19 Claude Opus 4 6.86%
  20. 20 GPT-5 high 6.86%
  21. 21 Claude Sonnet 4 4.90%
  22. 22 Kimi K2 Jul 2025 4.90%
  23. 23 Qwen3-Coder-480B-A35B 4.90%
  24. 24 Claude 3.5 Sonnet (October 2024) 4.60%
  25. 25 Gemini 2.5 Pro Jun 2025 3.92%
  26. 26 Claude 3.7 Sonnet 3.80%
  27. 27 o4-mini high 3.60%
  28. 28 GLM-4.5-Air 2.94%
  29. 29 o3-mini high 1.30%
  30. 30 GPT-4o Mar 2025 0%
#ModelScoreSettingDateSource
1 Claude Fable 5.1 88.2% xhigh, OpenHands Sep 2026 Official leaderboard
2 GPT-6 Astra 79.4% xhigh, OpenHands Sep 2026 Official leaderboard
3 Claude Fable 5 78.4% xhigh, OpenHands Jun 2026 Official leaderboard
4 GPT-5.6 Sol 76.5% xhigh, OpenHands Jul 2026 Official leaderboard
5 Claude Opus 4.8 47.1% xhigh, OpenHands May 2026 Official leaderboard
6 Claude Opus 4.7 44.1% high, OpenHands Apr 2026 Official leaderboard
7 Claude Opus 4.6 41.2% high, OpenHands Feb 2026 Official leaderboard
8 GPT-5.5 40.2% xhigh, OpenHands Apr 2026 Official leaderboard
9 Claude Sonnet 5 37.3% xhigh, OpenHands Jun 2026 Official leaderboard
10 GPT-5.4 31.4% xhigh, OpenHands Mar 2026 Official leaderboard
11 GPT-5.2 27.4% high, OpenHands Dec 2025 Official leaderboard
12 Claude Opus 4.5 26.5% OpenHands Nov 2025 Official leaderboard
13 Gemini 3.1 Pro 22.6% OpenHands Feb 2026 Official leaderboard
14 Gemini 3 Pro 18.6% OpenHands Nov 2025 Official leaderboard
15 Claude Sonnet 4.5 14.7% OpenHands Sep 2025 Official leaderboard
16 GPT-5.1 13.7% high, OpenHands Nov 2025 Official leaderboard
17 Gemini 3 Flash 9.80% OpenHands Dec 2025 Official leaderboard
18 o3 8.82% high, OpenHands Apr 2025 Official leaderboard
19 Claude Opus 4 6.86% OpenHands May 2025 Official leaderboard
20 GPT-5 6.86% high, OpenHands Aug 2025 Official leaderboard
21 Claude Sonnet 4 4.90% OpenHands May 2025 Official leaderboard
22 Kimi K2 Jul 2025 4.90% OpenHands Jul 2025 Official leaderboard
23 Qwen3-Coder-480B-A35B 4.90% OpenHands Jul 2025 Official leaderboard
24 Claude 3.5 Sonnet (October 2024) 4.60% OpenHands Oct 2024 Official leaderboard
25 Gemini 2.5 Pro Jun 2025 3.92% OpenHands Jun 2025 Official leaderboard
26 Claude 3.7 Sonnet 3.80% OpenHands Feb 2025 Official leaderboard
27 o4-mini 3.60% high, OpenHands Apr 2025 Official leaderboard
28 GLM-4.5-Air 2.94% OpenHands Jul 2025 Official leaderboard
29 o3-mini 1.30% high, OpenHands Jan 2025 Official leaderboard
30 GPT-4o Mar 2025 0% OpenHands Mar 2025 Official leaderboard

Epoch AI has scores for 30 models. We plot every source in the charts below.

10 models have held number one since its release in May 2025

Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and Epoch AI’s runs.

OpenAI Anthropic Google DeepMind
Own leaderboard Passed 85% in Sep 2026, by Claude Fable 5.1
Claude Fable 5.1 88.2%
Epoch AI runs Passed 85% in Sep 2026, by Claude Fable 5.1
Claude Fable 5.1 88.2%

The best score passed 85% in Sep 2026

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Run or collected by Epoch AI Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Fable 5.1 Claude Fable 5 Claude Opus 4.8 Claude Opus 4.6 GPT-5.2 Gemini 3 Pro o3 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) Claude Fable 5.1 Claude Fable 5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Coding

All 243 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Opus 5.5 93.9%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471