RecreationBench
Agents explore a running reference app and rebuild it, across 250 tasks on five platforms. Programmatic tests and a VLM judge score the result.
In the authors’ words
Computer-use agents (CUAs) have advanced along two separate lines: graphical interaction and software development through code and the command line. Real digital work requires both, interleaved rather than stacked end to end. We study hybrid CUAs that autonomously decide when to explore an interface, implement software, and run and visually verify their artifacts. We introduce RecreationWorld, a five-platform framework built around recreation: given a running reference, an agent must discover its behavior and build a faithful implementation with no prescribed workflow. RecreationWorld provides reproducible environments on Ubuntu, macOS, Windows, Android, and Web, plus a unified harness with native GUI control and coding tools. The running reference serves as an oracle for hidden behavioral tests, providing execution-grounded rewards. We scale trajectory generation with high-quality open-source applications. Models trained on these trajectories improve across five out-of-distribution coding and hybrid computer-use benchmarks and more frequently verify their rendered outputs, providing evidence of transfer beyond recreation. For held-out evaluation, we introduce RecreationBench, comprising 250 diverse tasks across domains and platforms. Reference-grounded programmatic and visual assertions cover action-conditioned outcomes at multiple interaction depths; each is validated on the reference and by human reviewers before the suite is frozen for automatic scoring. GPT-6 Astra leads at 58.1% overall, but passes all programmatic tests on just 2.8% of tasks. Agents reproduce static interface structure more reliably than interactions and computed outputs, while generated applications remain smaller and more monolithic than their references. We release the benchmark, environments, and test suites.
- Level
- Expert Specialists with years of training, often over hours of work
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Agent task Many steps in a terminal, browser, desktop, app, game or robot.
GPT-6 Astra leads with 58.1% among models released by Sep 2026
Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.
The newest model with a score here came out in Sep 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:
- Claude Opus 5.5
- GPT-6.1 Terra
- GPT-6.1 Sol
- Claude Sonnet 5.5
- and 11 more
- 1 GPT-6 Astra max 58.1%
- 2 Claude Opus 5 max 44.2%
- 3 GPT-5.6 Sol max 42.1%
- 4 Grok 4.6 xhigh 36.7%
- 5
Qwen3.8 Max (0902) xhigh 34.8%
- 6 Kimi K3 max 31.4%
- 7 Claude Opus 4.8 max 31.1%
- 8 GLM-5.3 max 24.4%
- 9
Gemini 3.7 Flash high 21.1%
- 10
Qwen3.7-Plus 9.15%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 58.1% | max | Sep 2026 | Official leaderboard |
| 2 | Claude Opus 5 | 44.2% | max | Jul 2026 | Official leaderboard |
| 3 | GPT-5.6 Sol | 42.1% | max | Jul 2026 | Official leaderboard |
| 4 | Grok 4.6 | 36.7% | xhigh | Aug 2026 | Official leaderboard |
| 5 | | 34.8% | xhigh | Sep 2026 | Official leaderboard |
| 6 | Kimi K3 | 31.4% | max | Jul 2026 | Official leaderboard |
| 7 | Claude Opus 4.8 | 31.1% | max | May 2026 | Official leaderboard |
| 8 | GLM-5.3 | 24.4% | max | Aug 2026 | Official leaderboard |
| 9 | | 21.1% | high | Aug 2026 | Official leaderboard |
| 10 | | 9.15% | Jun 2026 | Official leaderboard |
Labs report scores for 3 models in their cards. We plot every source in the charts below.
The best score, 58.1%, costs $116 a task
Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-6 Astra | max | 58.1% | $116 | ||
| Claude Opus 5 | max | 44.2% | $117 | ||
| GPT-5.6 Sol | max | 42.1% | $25 | ||
| Grok 4.6 | xhigh | 36.7% | $13 | ||
| | xhigh | 34.8% | $39 | ||
| Kimi K3 | max | 31.4% | $63 | ||
| Claude Opus 4.8 | max | 31.1% | $69 | ||
| GLM-5.3 | max | 24.4% | $71 | ||
| | high | 21.1% | |||
| | 9.15% | $1.23 |
2 models have held number one since its release in Sep 2026
Lab claims count from the day the card came out, independent runs from the model’s release. We draw one row each for the benchmark’s own leaderboard and what labs claim in their cards.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
One lab put it in a launch post in the last six months
Launch posts from every lab that has listed it, one tick per post. Tall colored ticks include it, and short gray ones leave it out.
| Date | Launch post | Score | As the post writes it |
|---|---|---|---|
| 26 Aug 2026 | 49.9 | Application recreation RecreationBench | |
| 14 Aug 2026 | 47.1 | Application recreation RecreationBench | |
| 3 Aug 2026 | 51.7 | RecreationBench |
More in Computer and browser use
- BrowseComp In use Hard-to-find facts that need persistent web browsing. Short verifiable answers graded for exact correctness. Claude Fable 5 93.3%
- ScreenSpot-Pro In use GUI grounding on high-resolution professional-software screenshots. The model must locate the target UI element. GPT-6 Astra 92.7%
- AndroidWorld In use Agents complete 116 tasks across 20 real Android apps in an emulator. Each task checks the device's system state to judge success. GPT-6 Astra 100%
- OSWorld 2.1 In use Agents complete 108 long-horizon workflows across 31 self-hosted websites and desktop apps, averaging 318 tool calls. Scored by binary completion and checkpoint partial credit. Claude Opus 5.5 81.8%
- WebArena-Verified In use Agents complete 812 audited WebArena tasks on self-hosted websites. Deterministic evaluators check the agent's answer and captured network traces.
Muse Spark 1.1 69%
- MobileWorld In use Agents complete 201 tasks across 20 mobile apps, including tasks that need user clarification or MCP tools. Backend checks verify success. Kimi K3 74.4%