StarSkirmish Bench
Each model gets one hour in its lab's own CLI to write a StarCraft: Brood War bot in C++. The score is its win rate against human-written bots.
v0.2 (2026-10-15) replaced v0.1 with new flagship models and first-party harnesses. Each model has five runs.
- Level
- Expert Specialists with years of training, often over hours of work
- Scored
- Ranked by matches Models play against each other, and the results become ratings.
- Answers
- Code A program or patch run against tests.
GPT-6 Astra leads with 57.3%
Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.
- 1 GPT-6 Astra max 57.3%
- 2 GPT-6.1 Sol max 45.8%
- 3
Gemini 4 Argon high 39.3% - 4 Claude Opus 5.5 max 38.6%
- 5 Claude Fable 5.1 max 37.2%
- 6 Claude Sonnet 5.5 max 25.8%
- 7
Muse Wonder Full max 21.4% - 8 Claude Haiku 5.5 max 21%
- 9 GPT-6 Luna max 13.1%
| # | Model | Score | Setting | Date | Source |
|---|---|---|---|---|---|
| 1 | GPT-6 Astra | 57.3% | max | Sep 2026 | Official leaderboard |
| 2 | GPT-6.1 Sol | 45.8% | max | Sep 2026 | Official leaderboard |
| 3 | Gemini 4 Argon | 39.3% | high | Sep 2026 | Official leaderboard |
| 4 | Claude Opus 5.5 | 38.6% | max | Sep 2026 | Official leaderboard |
| 5 | Claude Fable 5.1 | 37.2% | max | Sep 2026 | Official leaderboard |
| 6 | Claude Sonnet 5.5 | 25.8% | max | Sep 2026 | Official leaderboard |
| 7 | Muse Wonder Full | 21.4% | max | Oct 2026 | Official leaderboard |
| 8 | Claude Haiku 5.5 | 21% | max | Oct 2026 | Official leaderboard |
| 9 | GPT-6 Luna | 13.1% | max | Sep 2026 | Official leaderboard |
The best score, 57.3%, costs $18 a task
Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.
| Model | Effort | Harness | Score | Cost per task | Output tokens |
|---|---|---|---|---|---|
| GPT-6 Astra | max | 57.3% | $18 | ||
| GPT-6.1 Sol | max | 45.8% | $2.02 | ||
| Gemini 4 Argon | high | 39.3% | $3.63 | ||
| Claude Opus 5.5 | max | 38.6% | $5.12 | ||
| Claude Fable 5.1 | max | 37.2% | $7.78 | ||
| Claude Sonnet 5.5 | max | 25.8% | $2.38 | ||
| Muse Wonder Full | max | 21.4% | $5.87 | ||
| Claude Haiku 5.5 | max | 21% | $0.98 | ||
| GPT-6 Luna | max | 13.1% | $0.57 |
Number one over time
Lab claims count from the day the card came out, independent runs from the model’s release.
Scores over time
All the scores we have, one dot each, so a model tested at several settings appears more than once.
No lab model card reports it yet
We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.
More in Games and long tasks
- ARC-AGI-3 In use Interactive game environments where agents must explore, infer goals and rules, and act efficiently without instructions. GPT-6 Astra 100%
- MineDojo Fading Agents complete open-ended tasks in Minecraft, such as mining blocks and fighting mobs, from a large simulation suite. Programmatic checks score many tasks.
Seed2.0 Pro 49% - Orak Fading LLM agents play 12 popular video games across most genres through an MCP interface. Scores come from in-game progress or outcomes per title.
Nemotron 3 Nano 4B 22.9 - GameWorld Only in system cards Multimodal game agents play 34 browser games across 170 tasks. Outcomes are checked from the game state. Gemini 3 Flash 41.9%
- Swarm Bench Only in system cards Models orchestrate agent swarms on complex tasks that benefit from coordinated decomposition and parallel execution. Kimi K3 76.3%
- TextQuests Only in system cards Agents play 25 classic interactive fiction games. Measures game progress and harmful actions per 100 steps. Muse Spark 2.40