Benchmark Almanac

Vibe Code Bench v1

GPT-5.2 41.3% leads since Dec 2025 Not in model cards

Agents build web applications from scratch from a natural-language spec in OpenHands. The built apps are tested against functional requirements.

Board frozen 2026-02-20 with GPT 5.2 on top at 41.31%. Vals moved to v1.1 on 2026-03-02.

Made by Vals AI Released Nov 2025 Building whole projects Tasks kept private Last model added Feb 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Graded by a model A language model scores each response, usually against a rubric.
Answers
Code A program or patch run against tests.
vals-ai/VibeCodeBench-Openhands-Scaffold 5 stars Leaderboard
17
Models scored on its own leaderboard

GPT-5.2 leads with 41.3% among models released by Feb 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

The newest model with a score here came out in Feb 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

17 models. Scroll sideways for the rest.

  1. 41.3 GPT-5.2
  2. 36.1 Claude Opus 4.6
  3. 29.1 Claude Sonnet 4.6
  4. 24.6 GPT-5.1
  5. 22.6 Claude Sonnet 4.5
  6. 22.2 GPT-5.1-Codex-Max
  7. 20.6 Claude Opus 4.5
  8. 20.1 GPT-5
  9. 14.3 Gemini 3 Pro
  10. 13.1 GPT-5.1-Codex
  11. 11.7 Claude Haiku 4.5
  12. 4.87 GPT-5 mini
  13. 3.51 Qwen3-Max
  14. 3.09 GLM-4.6
  15. 0.40 Gemini 2.5 Pro
  16. 0.0 Grok 4 Fast
  17. 0.0 Grok 4.1 Fast
  1. 1 GPT-5.2 41.3%
  2. 2 Claude Opus 4.6 thinking 36.1%
  3. 3 Claude Sonnet 4.6 29.1%
  4. 4 GPT-5.1 24.6%
  5. 5 Claude Sonnet 4.5 thinking 22.6%
  6. 6 GPT-5.1-Codex-Max 22.2%
  7. 7 Claude Opus 4.5 thinking 20.6%
  8. 8 GPT-5 20.1%
  9. 9 Gemini 3 Pro 14.3%
  10. 10 GPT-5.1-Codex 13.1%
  11. 11 Claude Haiku 4.5 thinking 11.7%
  12. 12 GPT-5 mini 4.87%
  13. 13 Qwen3-Max 3.51%
  14. 14 GLM-4.6 3.09%
  15. 15 Gemini 2.5 Pro Jun 2025 0.40%
  16. 16 Grok 4 Fast 0%
  17. 17 Grok 4.1 Fast 0%
#ModelScoreSettingDateSource
1 GPT-5.2 41.3% OpenHands Dec 2025 Official leaderboard
2 Claude Opus 4.6 36.1% thinking, OpenHands Feb 2026 Official leaderboard
3 Claude Sonnet 4.6 29.1% OpenHands Feb 2026 Official leaderboard
4 GPT-5.1 24.6% OpenHands Nov 2025 Official leaderboard
5 Claude Sonnet 4.5 22.6% thinking, OpenHands Sep 2025 Official leaderboard
6 GPT-5.1-Codex-Max 22.2% OpenHands Nov 2025 Official leaderboard
7 Claude Opus 4.5 20.6% thinking, OpenHands Nov 2025 Official leaderboard
8 GPT-5 20.1% OpenHands Aug 2025 Official leaderboard
9 Gemini 3 Pro 14.3% OpenHands Nov 2025 Official leaderboard
10 GPT-5.1-Codex 13.1% OpenHands Nov 2025 Official leaderboard
11 Claude Haiku 4.5 11.7% thinking, OpenHands Oct 2025 Official leaderboard
12 GPT-5 mini 4.87% OpenHands Aug 2025 Official leaderboard
13 Qwen3-Max 3.51% OpenHands Sep 2025 Official leaderboard
14 GLM-4.6 3.09% OpenHands Sep 2025 Official leaderboard
15 Gemini 2.5 Pro Jun 2025 0.40% OpenHands Jun 2025 Official leaderboard
16 Grok 4 Fast 0% OpenHands Sep 2025 Official leaderboard
17 Grok 4.1 Fast 0% OpenHands Nov 2025 Official leaderboard

The best score, 41.3%, costs $27 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI xAI Anthropic Cost frontier One model at each effort level
0% 10% 20% 30% 40% 50% $0.10 $1.00 $10 $100 Cost per task, log scale GPT-5.2 Claude Opus 4.6, thinking GPT-5.1 GPT-5 GPT-5 mini Grok 4.1 Fast 0% 10% 20% 30% 40% 50% $0.10 $1.00 $10 $100 Cost per task, log scale GPT-5.2 Claude Opus 4.6, thinking
ModelEffortHarnessScoreCost per taskOutput tokens
GPT-5.2 OpenHands 41.3% $27
Claude Opus 4.6 thinking OpenHands 36.1% $11
Claude Sonnet 4.6 OpenHands 29.1% $12
GPT-5.1 OpenHands 24.6% $2.57
Claude Sonnet 4.5 thinking OpenHands 22.6% $6.66
GPT-5.1-Codex-Max OpenHands 22.2% $7.32
Claude Opus 4.5 thinking OpenHands 20.6% $33
GPT-5 OpenHands 20.1% $1.53
Gemini 3 Pro OpenHands 14.3% $6.42
GPT-5.1-Codex OpenHands 13.1% $3.80
Claude Haiku 4.5 thinking OpenHands 11.7% $1.60
GPT-5 mini OpenHands 4.87% $0.18
Qwen3-Max OpenHands 3.51% $6.02
GLM-4.6 OpenHands 3.09% $11
Gemini 2.5 Pro Jun 2025 OpenHands 0.40% $1.22
Grok 4 Fast OpenHands 0% $0.13
Grok 4.1 Fast OpenHands 0% $0.12

3 models have held number one since its release in Nov 2025

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic OpenAI
Own leaderboard Replaced by Bench, Mar 2026
GPT-5.2 41.3%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) GPT-5.2 GPT-5.1 GPT-5 Gemini 2.5 Pro 0% 20% 40% 60% 80% 100% 2025 2026 Near the ceiling (85%) GPT-5.2 GPT-5.1

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

Versions

We switch the top row to each new version when it comes out. Open it for one row per version. An old version stays in play while people still test it, so whoever first takes it past 85% gets credit for closing it out.

Anthropic OpenAI
Vibe Code Bench Passed 85% in Jun 2026, by Claude Fable 5
Claude Sonnet 5.5 92.4%
Vibe Code Bench v1 Replaced by Bench, Mar 2026
GPT-5.2 41.3%
Vibe Code Bench Passed 85% in Jun 2026, by Claude Fable 5
Claude Sonnet 5.5 92.4%
Replaced by Vibe Code Bench

More in Coding

All 243 in Coding
  • SWE-bench Verified In use 500 human-validated GitHub issues from 12 Python repos. Agent writes a patch that must pass hidden tests. Claude Opus 4.7 83.5%
  • SWE-bench Multilingual In use 300 GitHub-issue tasks across nine programming languages in SWE-bench format. Patch graded by repository tests. Claude Opus 5.5 93.9%
  • LiveCodeBench v6 In use Competitive programming problems collected after model cutoffs from LeetCode, AtCoder, Codeforces. Scored pass@1 on hidden tests. DeepSeek-V4-Pro 93.5%
  • DeepSWE In use 113 original long-horizon engineering tasks across 91 repos and five languages, written from scratch. Hand-written verifiers accept alternative solutions. GPT-6 Astra 74.1%
  • SciCode In use Scientist-curated research coding problems split into subproblems across natural sciences. Code checked against test cases. Seed2.1 Pro 59.8%
  • Codeforces rating In use Model solves recent Codeforces contest problems. An Elo-style rating is computed from simulated contest performance. DeepSeek V4.1 Flash 3,471