Benchmark Almanac

RWE-bench

GPT-6 Astra 37% leads since Oct 2026 Not in model cards

Agents do mechanical, electrical and aerospace design tasks, such as a crankshaft, dense PCB routing or turbine-vane cooling, with CAD and CAE tools. A deterministic verifier runs physics solvers and passes a design only if it meets every requirement.

Engineers on the team wrote each task with its verifier and a reference solution. Each model runs in its native harness at max effort. The tasks are never licensed for training. Launch coverage reports 20 tasks and 11 models, with GPT-6 Astra at 37%.

Made by Xplore AI Released Oct 2026 CAD models Tasks kept private Last model added Oct 2026 Scores refreshed 2 Nov 2026
Level
Expert Specialists with years of training, often over hours of work
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
Leaderboard xplore.ai
11
Models scored on its own leaderboard

GPT-6 Astra leads with 37%

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best. Every run is in the next chart.

  1. 37.0 GPT-6 Astra
  2. 29.0 Claude Opus 5.5
  3. 25.0 GPT-6.1 Sol
  4. 12.0 Gemini 4 Argon
  5. 7.00 Claude Fable 5.1
  6. 1.00 Kimi K3.1 Pro
  7. 0.0 Qwen 3.8 Max
  8. 0.0 GLM-5.3
  9. 0.0 DeepSeek V4.1 Flash
  10. 0.0 Grok 4.7
  11. 0.0 Muse Wonder
  1. 1 GPT-6 Astra max 37%
  2. 2 Claude Opus 5.5 max 29%
  3. 3 GPT-6.1 Sol max 25%
  4. 4 Gemini 4 Argon high 12%
  5. 5 Claude Fable 5.1 max 7.00%
  6. 6 Kimi K3.1 Pro max 1.00%
  7. 7 Qwen 3.8 Max xhigh 0%
  8. 8 GLM-5.3 max 0%
  9. 9 DeepSeek V4.1 Flash max 0%
  10. 10 Grok 4.7 xhigh 0%
  11. 11 Muse Wonder max 0%
#ModelScoreSettingDateSource
1 GPT-6 Astra 37% max, Codex Sep 2026 Official leaderboard
2 Claude Opus 5.5 29% max, Claude Code Sep 2026 Official leaderboard
3 GPT-6.1 Sol 25% max, Codex Sep 2026 Official leaderboard
4 Gemini 4 Argon 12% high, Gemini CLI Sep 2026 Official leaderboard
5 Claude Fable 5.1 7.00% max, Claude Code Sep 2026 Official leaderboard
6 Kimi K3.1 Pro 1.00% max, Kimi Code Oct 2026 Official leaderboard
7 Qwen 3.8 Max 0% xhigh, Qwen Coder Aug 2026 Official leaderboard
8 GLM-5.3 0% max, ZCode Aug 2026 Official leaderboard
9 DeepSeek V4.1 Flash 0% max, DSH Sep 2026 Official leaderboard
10 Grok 4.7 0% xhigh, Grok Build Sep 2026 Official leaderboard
11 Muse Wonder 0% max, Muse Code Oct 2026 Official leaderboard

The best score, 37%, costs $59 a task

Runs on the cost frontier score higher than any run that's cheaper. The costs are the leaderboard's own estimates, so compare them only within this benchmark.

OpenAI DeepSeek Anthropic Cost frontier One model at each effort level
0% 10% 20% 30% 40% $1.00 $2.00 $5.00 $10 $20 $50 $100 Cost per task, log scale GPT-6 Astra, max Claude Opus 5.5, max GPT-6.1 Sol, max DeepSeek V4.1 Flash, max 0% 10% 20% 30% 40% $1.00 $10 $100 Cost per task, log scale GPT-6 Astra, max Claude Opus 5.5, max
ModelEffortHarnessScoreCost per taskOutput tokens
GPT-6 Astra max Codex 37% $59 358,000
Claude Opus 5.5 max Claude Code 29% $58 739,000
GPT-6.1 Sol max Codex 25% $10 454,000
Gemini 4 Argon high Gemini CLI 12% $24 359,000
Claude Fable 5.1 max Claude Code 7.00% $98 652,000
Kimi K3.1 Pro max Kimi Code 1.00% $71 548,000
Qwen 3.8 Max xhigh Qwen Coder 0% $17 349,000
GLM-5.3 max ZCode 0% $23 1,029,000
DeepSeek V4.1 Flash max DSH 0% $1.03 424,000
Grok 4.7 xhigh Grok Build 0% $66 517,000
Muse Wonder max Muse Code 0% $28 644,000

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI
Own leaderboard 1 leader
GPT-6 Astra 37%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) GPT-6 Astra Claude Fable 5.1 Qwen 3.8 Max 0% 20% 40% 60% 80% 100% 2027 Near the ceiling (85%) GPT-6 Astra Claude Fable 5.1

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Engineering and CAD

All 18 in Engineering and CAD
  • BenchCAD In use Programmatic CAD: 17,900 execution-verified CadQuery programs over 106 industrial part families. Four matched tasks scored on geometry. GPT-6 Astra 95.9%
  • EEBench In use 13 electrical-engineering circuit and PCB design tasks graded by deterministic build checks, simulation and BOM constraints. No LLM judge. Claude Opus 5.5 75%
  • Parametric CAD Bench In use Agents author editable parametric FreeCAD part designs, graded against held-back references for geometry and spec consistency. Claude Opus 5.5 61%
  • CADGenBench Only in system cards Agents construct CAD models from design prompts. The generation split is graded for executability and geometric correctness. Claude Fable 5 45.1%
  • 3DCodeBench Only in system cards Agentic procedural 3D modeling via code: author engine-ready 3D assets through software APIs. Graded for executability and shape fidelity. Grok 4.6 54%
  • PartBench Only in system cards Parametric mechanical part design and CAD-style generation, graded for geometric and manufacturing fidelity. Grok 4.6 58.5%