Benchmark Almanac

ArtifactArena

GPT-6 Astra 1,539 leads since Sep 2026 Not in model cards

Models design robot bodies, controllers and strategy for a simulated sumo-style arena. Artifacts fight head-to-head and models get Elo ratings.

Built to resist saturation: new designs face the best existing robots, and community bots can be submitted. Three harnesses: one-shot, verifier-guided refinement, open-ended design lab. Models did worse with full autonomy than with verifier-guided refinement. No arXiv ID yet.

Made by MIT CSAIL + MIT Media Lab + NYU + UCSF Released Sep 2026 Simulated robots All tasks public Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Ranked by matches Models play against each other, and the results become ratings.
Answers
Code A program or patch run against tests.
ArtifactArena/design-lab-harness 0 stars Paper Leaderboard
21
Models scored on its own leaderboard

GPT-6 Astra leads with 1,539 among models released by Sep 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in Sep 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

21 models. Scroll sideways for the rest. Bars start at 600.

  1. 1,539 GPT-6 Astra
  2. 1,310 Claude Fable 5.1
  3. 1,223 Gemini 3.8 Flash
  4. 1,189 Claude Fable 5
  5. 1,179 GPT-5.4
  6. 1,173 GPT-5.3 Codex
  7. 1,165 GPT-5.6 Sol
  8. 1,164 Kimi K3
  9. 1,136 Grok 4.6
  10. 1,093 GPT-5.6 Terra
  11. 1,089 Gemini 3.1 Pro
  12. 1,055 Qwen3.8-2.4T-A95B
  13. 1,051 GLM-5.2
  14. 986 Claude Sonnet 5
  15. 984 GPT-5.6 Luna
  16. 978 GLM-5.3
  17. 967 GPT-5.5
  18. 957 Grok 4.5
  19. 929 Claude Opus 4.8
  20. 765 DeepSeek-V4-Pro
  21. 745 Grok 4.20

Bars start at 600.

  1. 1 GPT-6 Astra 1,539
  2. 2 Claude Fable 5.1 high 1,310
  3. 3 Gemini 3.8 Flash high 1,223
  4. 4 Claude Fable 5 high 1,189
  5. 5 GPT-5.4 1,179
  6. 6 GPT-5.3 Codex high 1,173
  7. 7 GPT-5.6 Sol 1,165
  8. 8 Kimi K3 high 1,164
  9. 9 Grok 4.6 high 1,136
  10. 10 GPT-5.6 Terra 1,093
  11. 11 Gemini 3.1 Pro high 1,089
  12. 12 Qwen3.8-2.4T-A95B thinking 1,055
  13. 13 GLM-5.2 high 1,051
  14. 14 Claude Sonnet 5 high 986
  15. 15 GPT-5.6 Luna 984
  16. 16 GLM-5.3 high 978
  17. 17 GPT-5.5 967
  18. 18 Grok 4.5 high 957
  19. 19 Claude Opus 4.8 high 929
  20. 20 DeepSeek-V4-Pro thinking 765
  21. 21 Grok 4.20 reasoning 745
#ModelScoreSettingDateSource
1 GPT-6 Astra 1,539 Verifier-Grounded Refinement Sep 2026 Official leaderboard
2 Claude Fable 5.1 1,310 high, Verifier-Grounded Refinement Sep 2026 Official leaderboard
3 Gemini 3.8 Flash 1,223 high, Verifier-Grounded Refinement Sep 2026 Official leaderboard
4 Claude Fable 5 1,189 high, Design Lab Jun 2026 Official leaderboard
5 GPT-5.4 1,179 Sampling Mar 2026 Official leaderboard
6 GPT-5.3 Codex 1,173 high, Verifier-Grounded Refinement Feb 2026 Official leaderboard
7 GPT-5.6 Sol 1,165 Sampling Jul 2026 Official leaderboard
8 Kimi K3 1,164 high, Verifier-Grounded Refinement Jul 2026 Official leaderboard
9 Grok 4.6 1,136 high, Verifier-Grounded Refinement Aug 2026 Official leaderboard
10 GPT-5.6 Terra 1,093 Verifier-Grounded Refinement Jul 2026 Official leaderboard
11 Gemini 3.1 Pro 1,089 high, Sampling Feb 2026 Official leaderboard
12 Qwen3.8-2.4T-A95B 1,055 thinking, Verifier-Grounded Refinement Aug 2026 Official leaderboard
13 GLM-5.2 1,051 high, Verifier-Grounded Refinement Jun 2026 Official leaderboard
14 Claude Sonnet 5 986 high, Verifier-Grounded Refinement Jun 2026 Official leaderboard
15 GPT-5.6 Luna 984 Verifier-Grounded Refinement Jul 2026 Official leaderboard
16 GLM-5.3 978 high, Design Lab Aug 2026 Official leaderboard
17 GPT-5.5 967 Sampling Apr 2026 Official leaderboard
18 Grok 4.5 957 high, Verifier-Grounded Refinement Jul 2026 Official leaderboard
19 Claude Opus 4.8 929 high, Verifier-Grounded Refinement May 2026 Official leaderboard
20 DeepSeek-V4-Pro 765 thinking, Verifier-Grounded Refinement Apr 2026 Official leaderboard
21 Grok 4.20 745 reasoning, Verifier-Grounded Refinement Feb 2026 Official leaderboard

2 models have held number one since its release in Sep 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic OpenAI
Own leaderboard Not tested since Sep 2026
GPT-6 Astra 1,539

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard
500 750 1,000 1,250 1,500 2026 500 750 1,000 1,250 1,500 2026

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Robotics

All 86 in Robotics
  • ERQA In use 400 multiple-choice visual questions from robot and egocentric footage on spatial, trajectory, action and state reasoning. Accuracy. Gemini Robotics ER 2 78.5%
  • RoboLab In use Zero-shot DROID Franka policies in Isaac Lab on 120 tabletop tasks at three instruction levels, 10 episodes each. Success rate. GR00T N1.7 8.58%
  • Human proximity (Google DeepMind robotics) In use Models judge from robot camera views whether a human is within 1 meter, a spatial awareness check for safe robot behavior. Scored by accuracy. Gemini Robotics ER 2 93%
  • Moment finding (Google DeepMind) In use Models find the moment in a robot video when an event happens. Scored by accuracy and mean absolute time distance in seconds. Gemini Robotics ER 2 91.3%
  • Progress classification (Google DeepMind) In use Models assign each frame of a robot video to one of five task-progress bands from 0 to 100 percent. Scored by accuracy. Gemini Robotics ER 2 57.4%
  • LIBERO Fading Simulated Franka tabletop tasks in four 10-task suites. Policies fine-tuned on 50 demos per task. Success rate over ~50 trials. MolmoAct 2 97.2%