Benchmark Almanac

Build-A-Chip-Bench

Claude Fable 5.1 16.6% leads since Oct 2026 Not in model cards

Agents take a chip from workload specification to a routed GDS layout with open-source tools. Each design must pass hidden verification, routability and timing gates.

20 tasks, such as an inference accelerator for a small language model. 8 models in the Fireworks index.

Made by ChipAgents (with NVIDIA contributions) Released Oct 2026 Electronics Tasks kept private Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
Research frontier Researchers, on open problems or tasks that take days
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.
chipagents.ai
5
Models scored on its own leaderboard

Claude Fable 5.1 leads with 16.6% among models released by Sep 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in Sep 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 16.6 Claude Fable 5.1
  2. 8.70 GPT-6 Astra
  3. 0.80 Gemini 3.8 Flash
  4. 0.20 Kimi K3
  5. 0.0 GLM-5.3
  1. 1 Claude Fable 5.1 16.6%
  2. 2 GPT-6 Astra 8.70%
  3. 3 Gemini 3.8 Flash 0.80%
  4. 4 Kimi K3 0.20%
  5. 5 GLM-5.3 0%
#ModelScoreSettingDateSource
1 Claude Fable 5.1 16.6% OpenCode Sep 2026 Official leaderboard
2 GPT-6 Astra 8.70% OpenCode Sep 2026 Official leaderboard
3 Gemini 3.8 Flash 0.80% OpenCode Sep 2026 Official leaderboard
4 Kimi K3 0.20% OpenCode Jul 2026 Official leaderboard
5 GLM-5.3 0% OpenCode Aug 2026 Official leaderboard

Number one over time

Lab claims count from the day the card came out, independent runs from the model’s release.

Anthropic
Own leaderboard 1 leader
Claude Fable 5.1 16.6%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Best so far
0% 20% 40% 60% 80% 100% Near the ceiling (85%) Claude Fable 5.1 Kimi K3 0% 20% 40% 60% 80% 100% Near the ceiling (85%) Claude Fable 5.1 Kimi K3

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Engineering and CAD

All 18 in Engineering and CAD
  • BenchCAD In use Programmatic CAD: 17,900 execution-verified CadQuery programs over 106 industrial part families. Four matched tasks scored on geometry. GPT-6 Astra 95.9%
  • EEBench In use 13 electrical-engineering circuit and PCB design tasks graded by deterministic build checks, simulation and BOM constraints. No LLM judge. Claude Opus 5.5 75%
  • Parametric CAD Bench In use Agents author editable parametric FreeCAD part designs, graded against held-back references for geometry and spec consistency. Claude Opus 5.5 61%
  • CADGenBench Only in system cards Agents construct CAD models from design prompts. The generation split is graded for executability and geometric correctness. Claude Fable 5 45.1%
  • 3DCodeBench Only in system cards Agentic procedural 3D modeling via code: author engine-ready 3D assets through software APIs. Graded for executability and shape fidelity. Grok 4.6 54%
  • PartBench Only in system cards Parametric mechanical part design and CAD-style generation, graded for geometric and manufacturing fidelity. Grok 4.6 58.5%