Benchmark Almanac

Code Migration

Claude Sonnet 5.5 69.8% leads since Sep 2026 Not in model cards

Models port a working program into another language offline and ship a Dockerfile. A hidden behavior-test suite the model never sees scores the port, after an anti-cheat check.

Ran on the mini-SWE-agent harness. 30 open-source CLI repositories in five languages, each moved to four targets, plus 10 COBOL-to-Java repositories. Superseded by v1.1 in October 2026.

Made by Vals AI Released Jun 2026 Fixing real repositories Tasks kept private Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Code A program or patch run against tests.
Leaderboard vals.ai
78
Models scored on its own leaderboard

Claude Sonnet 5.5 leads with 69.8%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

78 models. Scroll sideways for the rest.

  1. 69.8 Claude Sonnet 5.5
  2. 68.2 Gemini 4 Argon
  3. 67.7 GPT-6 Astra
  4. 66.7 Claude Opus 5.5
  5. 65.1 GPT-6.1 Sol
  6. 62.5 Muse Wonder
  7. 61.0 Muse Wonder Full
  8. 57.5 Claude Opus 5
  9. 57.2 GPT-6 Sol
  10. 55.1 Claude Fable 5
  11. 54.6 Claude Fable 5.1
  12. 52.9 GPT-5.6 Sol
  13. 47.8 GPT-5.6 Terra
  14. 47.4 Tencent Hy4 preview
  15. 47.4 Muse Spark 1.3
  16. 47.3 Claude Opus 4.8
  17. 45.6 DeepSeek V4.1 Flash
  18. 45.2 GPT-5.5
  19. 44.8 Grok 4.7
  20. 44.6 Grok 4.6
  21. 44.6 Claude Haiku 5.5
  22. 44.5 GPT-5.6 Luna
  23. 44.4 Claude Sonnet 5
  24. 44.2 GLM-5.3
  25. 44.2 DeepSeek V4.1 Flash 1017
  26. 43.9 Claude Opus 4.7
  27. 43.0 MiMo-V2.6 Pro
  28. 42.5 GPT-6 Luna
  29. 41.5 DeepSeek V4 Pro 0813
  30. 40.9 MiMo-V2.6 Flash
  31. 39.9 Claude Sonnet 4.6
  32. 39.0 Step 5 Preview
  33. 38.6 DeepSeek V4 Flash 0731
  34. 37.9 GLM-5.2
  35. 36.6 Grok 4.5
  36. 36.5 Gemini 3.8 Flash
  37. 35.0 GPT-5.4
  38. 34.8 Gemini 3.7 Flash
  39. 32.5 Ember-1
  40. 31.1 Muse Spark 1.1
  41. 30.9 Gemini 3.6 Flash
  42. 30.6 Mistral Large 4
  43. 29.9 Muse Spark 1.2
  44. 27.8 Kimi K2.6
  45. 26.8 Gemini 3.5 Flash
  46. 26.2 DeepSeek-V4-Pro
  47. 25.8 GLM-5.1
  48. 25.4 Kimi K2.7 Code
  49. 24.0 Qwen 3.8 Max
  50. 21.6 MiMo-V2.5-Pro
  51. 21.2 Muse Spark Open
  52. 20.5 GLM-5.3-Flash
  53. 19.9 MiniMax-M3
  54. 17.3 Gemini 3.1 Pro
  55. 16.1 Kimi K3
  56. 14.5 GPT-5.4 Nano
  57. 14.2 MiMo-V2.5
  58. 14.2 Qwen 3.8 27B
  59. 13.7 Inkling-Small
  60. 13.1 Qwen3.7-Max
  61. 12.9 GPT-5.4 Mini
  62. 12.9 Qwen3.7-Plus
  63. 11.8 Inkling
  64. 11.1 Qwen 3.6 Plus
  65. 8.69 MiniMax-M2.7
  66. 7.55 Claude Haiku 4.5
  67. 6.96 Kimi K2.5
  68. 6.79 Grok 4.3
  69. 6.39 Gemini 3 Flash
  70. 6.15 Gemini 3.5 Flash-Lite
  71. 5.13 Mistral Medium 3.5
  72. 4.91 Nemotron 3 Ultra
  73. 4.61 Gemini 3.1 Flash-Lite
  74. 4.52 Mercury 2.5
  75. 2.95 Nemotron 3.5 Lightning 30B-A3B
  76. 1.82 Ling-3.0-flash-Fin
  77. 0.31 Grok 4.20
  78. 0.0 Ling-3.0-flash
  1. 1 Claude Sonnet 5.5 69.8%
  2. 2 Gemini 4 Argon 68.2%
  3. 3 GPT-6 Astra 67.7%
  4. 4 Claude Opus 5.5 66.7%
  5. 5 GPT-6.1 Sol 65.1%
  6. 6 Muse Wonder 62.5%
  7. 7 Muse Wonder Full 61%
  8. 8 Claude Opus 5 57.5%
  9. 9 GPT-6 Sol 57.2%
  10. 10 Claude Fable 5 55.1%
  11. 11 Claude Fable 5.1 54.6%
  12. 12 GPT-5.6 Sol 52.9%
  13. 13 GPT-5.6 Terra 47.8%
  14. 14 Tencent Hy4 preview 47.4%
  15. 15 Muse Spark 1.3 max 47.4%
  16. 16 Claude Opus 4.8 47.3%
  17. 17 DeepSeek V4.1 Flash 45.6%
  18. 18 GPT-5.5 45.2%
  19. 19 Grok 4.7 44.8%
  20. 20 Grok 4.6 44.6%
  21. 21 Claude Haiku 5.5 44.6%
  22. 22 GPT-5.6 Luna 44.5%
  23. 23 Claude Sonnet 5 44.4%
  24. 24 GLM-5.3 44.2%
  25. 25 DeepSeek V4.1 Flash 1017 44.2%
  26. 26 Claude Opus 4.7 43.9%
  27. 27 MiMo-V2.6 Pro 43%
  28. 28 GPT-6 Luna 42.5%
  29. 29 DeepSeek V4 Pro 0813 41.5%
  30. 30 MiMo-V2.6 Flash 40.9%
  31. 31 Claude Sonnet 4.6 39.9%
  32. 32 Step 5 Preview 39%
  33. 33 DeepSeek V4 Flash 0731 38.6%
  34. 34 GLM-5.2 37.9%
  35. 35 Grok 4.5 36.6%
  36. 36 Gemini 3.8 Flash 36.5%
  37. 37 GPT-5.4 xhigh 35%
  38. 38 Gemini 3.7 Flash 34.8%
  39. 39 Ember-1 32.5%
  40. 40 Muse Spark 1.1 31.1%
  41. 41 Gemini 3.6 Flash 30.9%
  42. 42 Mistral Large 4 30.6%
  43. 43 Muse Spark 1.2 29.9%
  44. 44 Kimi K2.6 27.8%
  45. 45 Gemini 3.5 Flash 26.8%
  46. 46 DeepSeek-V4-Pro 26.2%
  47. 47 GLM-5.1 25.8%
  48. 48 Kimi K2.7 Code 25.4%
  49. 49 Qwen 3.8 Max 24%
  50. 50 MiMo-V2.5-Pro 21.6%
  51. 51 Muse Spark Open 21.2%
  52. 52 GLM-5.3-Flash 20.5%
  53. 53 MiniMax-M3 19.9%
  54. 54 Gemini 3.1 Pro 17.3%
  55. 55 Kimi K3 16.1%
  56. 56 GPT-5.4 Nano 14.5%
  57. 57 MiMo-V2.5 14.2%
  58. 58 Qwen 3.8 27B 14.2%
  59. 59 Inkling-Small 13.7%
  60. 60 Qwen3.7-Max 13.1%
  61. 61 GPT-5.4 Mini 12.9%
  62. 62 Qwen3.7-Plus 12.9%
  63. 63 Inkling 11.8%
  64. 64 Qwen 3.6 Plus 11.1%
  65. 65 MiniMax-M2.7 8.69%
  66. 66 Claude Haiku 4.5 thinking 7.55%
  67. 67 Kimi K2.5 thinking 6.96%
  68. 68 Grok 4.3 6.79%
  69. 69 Gemini 3 Flash 6.39%
  70. 70 Gemini 3.5 Flash-Lite 6.15%
  71. 71 Mistral Medium 3.5 5.13%
  72. 72 Nemotron 3 Ultra 4.91%
  73. 73 Gemini 3.1 Flash-Lite 4.61%
  74. 74 Mercury 2.5 4.52%
  75. 75 Nemotron 3.5 Lightning 30B-A3B 2.95%
  76. 76 Ling-3.0-flash-Fin 1.82%
  77. 77 Grok 4.20 0.31%
  78. 78 Ling-3.0-flash 0%
#ModelScoreSettingDateSource
1 Claude Sonnet 5.5 69.8% Sep 2026 Official leaderboard
2 Gemini 4 Argon 68.2% Sep 2026 Official leaderboard
3 GPT-6 Astra 67.7% Sep 2026 Official leaderboard
4 Claude Opus 5.5 66.7% Sep 2026 Official leaderboard
5 GPT-6.1 Sol 65.1% Sep 2026 Official leaderboard
6 Muse Wonder 62.5% Oct 2026 Official leaderboard
7 Muse Wonder Full 61% Oct 2026 Official leaderboard
8 Claude Opus 5 57.5% Jul 2026 Official leaderboard
9 GPT-6 Sol 57.2% Sep 2026 Official leaderboard
10 Claude Fable 5 55.1% Jun 2026 Official leaderboard
11 Claude Fable 5.1 54.6% Sep 2026 Official leaderboard
12 GPT-5.6 Sol 52.9% Jul 2026 Official leaderboard
13 GPT-5.6 Terra 47.8% Jul 2026 Official leaderboard
14 Tencent Hy4 preview 47.4% Aug 2026 Official leaderboard
15 Muse Spark 1.3 47.4% max Sep 2026 Official leaderboard
16 Claude Opus 4.8 47.3% May 2026 Official leaderboard
17 DeepSeek V4.1 Flash 45.6% Sep 2026 Official leaderboard
18 GPT-5.5 45.2% Apr 2026 Official leaderboard
19 Grok 4.7 44.8% Sep 2026 Official leaderboard
20 Grok 4.6 44.6% Aug 2026 Official leaderboard
21 Claude Haiku 5.5 44.6% Oct 2026 Official leaderboard
22 GPT-5.6 Luna 44.5% Jul 2026 Official leaderboard
23 Claude Sonnet 5 44.4% Jun 2026 Official leaderboard
24 GLM-5.3 44.2% Aug 2026 Official leaderboard
25 DeepSeek V4.1 Flash 1017 44.2% Oct 2026 Official leaderboard
26 Claude Opus 4.7 43.9% Apr 2026 Official leaderboard
27 MiMo-V2.6 Pro 43% Sep 2026 Official leaderboard
28 GPT-6 Luna 42.5% Sep 2026 Official leaderboard
29 DeepSeek V4 Pro 0813 41.5% Aug 2026 Official leaderboard
30 MiMo-V2.6 Flash 40.9% Sep 2026 Official leaderboard
31 Claude Sonnet 4.6 39.9% Feb 2026 Official leaderboard
32 Step 5 Preview 39% Sep 2026 Official leaderboard
33 DeepSeek V4 Flash 0731 38.6% Jul 2026 Official leaderboard
34 GLM-5.2 37.9% Jun 2026 Official leaderboard
35 Grok 4.5 36.6% Jul 2026 Official leaderboard
36 Gemini 3.8 Flash 36.5% Sep 2026 Official leaderboard
37 GPT-5.4 35% xhigh Mar 2026 Official leaderboard
38 Gemini 3.7 Flash 34.8% Aug 2026 Official leaderboard
39 Ember-1 32.5% Sep 2026 Official leaderboard
40 Muse Spark 1.1 31.1% Jul 2026 Official leaderboard
41 Gemini 3.6 Flash 30.9% Jul 2026 Official leaderboard
42 Mistral Large 4 30.6% Oct 2026 Official leaderboard
43 Muse Spark 1.2 29.9% Aug 2026 Official leaderboard
44 Kimi K2.6 27.8% Apr 2026 Official leaderboard
45 Gemini 3.5 Flash 26.8% May 2026 Official leaderboard
46 DeepSeek-V4-Pro 26.2% Apr 2026 Official leaderboard
47 GLM-5.1 25.8% Apr 2026 Official leaderboard
48 Kimi K2.7 Code 25.4% Jun 2026 Official leaderboard
49 Qwen 3.8 Max 24% Aug 2026 Official leaderboard
50 MiMo-V2.5-Pro 21.6% Apr 2026 Official leaderboard
51 Muse Spark Open 21.2% Oct 2026 Official leaderboard
52 GLM-5.3-Flash 20.5% Aug 2026 Official leaderboard
53 MiniMax-M3 19.9% Jun 2026 Official leaderboard
54 Gemini 3.1 Pro 17.3% Feb 2026 Official leaderboard
55 Kimi K3 16.1% Jul 2026 Official leaderboard
56 GPT-5.4 Nano 14.5% Mar 2026 Official leaderboard
57 MiMo-V2.5 14.2% Apr 2026 Official leaderboard
58 Qwen 3.8 27B 14.2% Aug 2026 Official leaderboard
59 Inkling-Small 13.7% Jul 2026 Official leaderboard
60 Qwen3.7-Max 13.1% May 2026 Official leaderboard
61 GPT-5.4 Mini 12.9% Mar 2026 Official leaderboard
62 Qwen3.7-Plus 12.9% Jun 2026 Official leaderboard
63 Inkling 11.8% Jul 2026 Official leaderboard
64 Qwen 3.6 Plus 11.1% Mar 2026 Official leaderboard
65 MiniMax-M2.7 8.69% Mar 2026 Official leaderboard
66 Claude Haiku 4.5 7.55% thinking Oct 2025 Official leaderboard
67 Kimi K2.5 6.96% thinking Jan 2026 Official leaderboard
68 Grok 4.3 6.79% Apr 2026 Official leaderboard
69 Gemini 3 Flash 6.39% Dec 2025 Official leaderboard
70 Gemini 3.5 Flash-Lite 6.15% Jul 2026 Official leaderboard
71 Mistral Medium 3.5 5.13% Apr 2026 Official leaderboard
72 Nemotron 3 Ultra 4.91% Jun 2026 Official leaderboard
73 Gemini 3.1 Flash-Lite 4.61% Mar 2026 Official leaderboard
74 Mercury 2.5 4.52% Sep 2026 Official leaderboard
75 Nemotron 3.5 Lightning 30B-A3B 2.95% Aug 2026 Official leaderboard
76 Ling-3.0-flash-Fin 1.82% Sep 2026 Official leaderboard
77 Grok 4.20 0.31% Feb 2026 Official leaderboard
78 Ling-3.0-flash 0% Aug 2026 Official leaderboard

The best score, 69.8%, costs $76 a task

Runs on the cost frontier score higher than any run that's cheaper. GPT-6.1 Sol comes within 4.7 points of the best for $6.51 a task, 12 times cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

Google DeepMind Xiaomi Corp Tencent OpenAI DeepSeek Anthropic Cost frontier One model at each effort level
0% 20% 40% 60% 80% $0.01 $0.10 $1.00 $10 $100 $1,000 Cost per task, log scale Claude Sonnet 5.5 GPT-6.1 Sol Tencent Hy4 preview MiMo-V2.6 Pro MiMo-V2.5-Pro MiMo-V2.5 Gemini 3.1 Flash-Lite 0% 20% 40% 60% 80% $0.01 $0.10 $1.00 $10 $100 $1,000 Cost per task, log scale Claude Sonnet 5.5 GPT-6.1 Sol
ModelEffortHarnessScoreCost per taskOutput tokens
Claude Sonnet 5.5 69.8% $76
Gemini 4 Argon 68.2% $58
GPT-6 Astra 67.7% $44
Claude Opus 5.5 66.7% $113
GPT-6.1 Sol 65.1% $6.51
Muse Wonder 62.5% $70
Muse Wonder Full 61% $38
Claude Opus 5 57.5% $61
GPT-6 Sol 57.2% $16
Claude Fable 5 55.1% $112
Claude Fable 5.1 54.6% $71
GPT-5.6 Sol 52.9% $25
GPT-5.6 Terra 47.8% $8.13
Tencent Hy4 preview 47.4% $3.41
Muse Spark 1.3 max 47.4% $14
Claude Opus 4.8 47.3% $31
DeepSeek V4.1 Flash 45.6% $0.94
GPT-5.5 45.2% $6.44
Grok 4.7 44.8% $37
Grok 4.6 44.6% $15
Claude Haiku 5.5 44.6% $6.23
GPT-5.6 Luna 44.5% $1.88
Claude Sonnet 5 44.4% $35
GLM-5.3 44.2% $25
DeepSeek V4.1 Flash 1017 44.2% $1.86
Claude Opus 4.7 43.9% $36
MiMo-V2.6 Pro 43% $0.69
GPT-6 Luna 42.5% $0.60
DeepSeek V4 Pro 0813 41.5% $19
MiMo-V2.6 Flash 40.9% $0.49
Claude Sonnet 4.6 39.9% $39
Step 5 Preview 39% $7.61
DeepSeek V4 Flash 0731 38.6% $3.61
GLM-5.2 37.9% $13
Grok 4.5 36.6% $3.37
Gemini 3.8 Flash 36.5% $18
GPT-5.4 xhigh 35% $7.76
Gemini 3.7 Flash 34.8% $21
Ember-1 32.5% $51
Muse Spark 1.1 31.1% $4.56
Gemini 3.6 Flash 30.9% $8.93
Mistral Large 4 30.6% $25
Muse Spark 1.2 29.9% $3.78
Kimi K2.6 27.8% $5.11
Muse Spark 1.3 27.6% $4.62
Gemini 3.5 Flash 26.8% $5.73
DeepSeek-V4-Pro 26.2% $1.07
GLM-5.1 25.8% $5.70
Kimi K2.7 Code 25.4% $5.04
Qwen 3.8 Max 24% $11
MiMo-V2.5-Pro 21.6% $0.24
Muse Spark Open 21.2% $9.04
GLM-5.3-Flash 20.5% $3.13
MiniMax-M3 19.9% $7.07
Gemini 3.1 Pro 17.3% $1.61
Kimi K3 16.1% $14
GPT-5.4 Nano 14.5% $0.42
MiMo-V2.5 14.2% $0.10
Qwen 3.8 27B 14.2% $22
Inkling-Small 13.7% $0.60
Qwen3.7-Max 13.1% $1.93
GPT-5.4 Mini 12.9% $1.95
Qwen3.7-Plus 12.9% $0.42
Inkling 11.8% $3.01
Qwen 3.6 Plus 11.1% $4.78
MiniMax-M2.7 8.69% $0.59
Claude Haiku 4.5 thinking 7.55% $0.98
Kimi K2.5 thinking 6.96% $0.93
Grok 4.3 6.79% $0.42
Gemini 3 Flash 6.39% $0.34
Gemini 3.5 Flash-Lite 6.15% $0.53
Mistral Medium 3.5 5.13% $19
Nemotron 3 Ultra 4.91%
Gemini 3.1 Flash-Lite 4.61% $0.05
Mercury 2.5 4.52% $0.63
Nemotron 3.5 Lightning 30B-A3B 2.95% $0.27
Ling-3.0-flash-Fin 1.82% $0.10
Grok 4.20 0.31% $0.29
Ling-3.0-flash 0% $0.06

5 models have held number one since its release in Jun 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic OpenAI
Own leaderboard 5 leaders
Claude Sonnet 5.5 69.8%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Sonnet 5.5 Claude Opus 5 Claude Opus 4.8 Claude Sonnet 4.6 Claude Haiku 4.5 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Sonnet 5.5 Claude Opus 5

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic OpenAI
Code Migration v1.1 Passed 85% in Oct 2026, by Claude Sonnet 5.5
Claude Sonnet 5.5 86.9%
Code Migration 5 leaders
Claude Sonnet 5.5 69.8%
Code Migration v1.1 Passed 85% in Oct 2026, by Claude Sonnet 5.5
Claude Sonnet 5.5 86.9%
Replaced by Code Migration v1.1

More in Coding

All 243 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Opus 5.5 93.9%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471