Benchmark Almanac

CADGenBench

Claude Fable 5 45.1% leads since Jun 2026 Only in system cards

Agents construct CAD models from design prompts. The generation split is graded for executability and geometric correctness.

Made by Mecado + Hugging Face Released May 2026 CAD models Last in a system card Sep 2026 Last model added Sep 2026 Scores refreshed 2 Nov 2026
Level
University Undergraduates and entry-level professionals
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Code A program or patch run against tests.
huggingface/cadgenbench 149 stars Leaderboard
11
Models scored on its own leaderboard
2
System cards report it, from 1 lab

Claude Fable 5 leads with 45.1% among models released by Jun 2026

Best score per model from the benchmark's own leaderboard, read on 2 Nov 2026. Where a model ran with several settings or harnesses, we keep its best.

The newest model with a score here came out in Jun 2026, and nobody has tested a newer one. Among the 20 highest-ranked models on the capability index, these have no score yet:

  1. 45.1 Claude Fable 5
  2. 38.7 GPT-5.5 Pro
  3. 36.9 Claude Opus 4.7
  4. 36.0 GPT-5.5
  5. 34.5 Claude Opus 4.8
  6. 31.1 Gemini 3.1 Pro
  7. 30.9 Claude Opus 4.6
  8. 28.3 Claude Sonnet 4.6
  9. 24.9 Gemini 3.1 Flash-Lite
  10. 18.3 GLM-4.6V
  11. 16.8 Qwen3-VL-235B-A22B-Instruct
  1. 1 Claude Fable 5 45.1%
  2. 2 GPT-5.5 Pro 38.7%
  3. 3 Claude Opus 4.7 36.9%
  4. 4 GPT-5.5 36%
  5. 5 Claude Opus 4.8 34.5%
  6. 6 Gemini 3.1 Pro 31.1%
  7. 7 Claude Opus 4.6 30.9%
  8. 8 Claude Sonnet 4.6 28.3%
  9. 9 Gemini 3.1 Flash-Lite 24.9%
  10. 10 GLM-4.6V 18.3%
  11. 11 Qwen3-VL-235B-A22B-Instruct 16.8%
#ModelScoreSettingDateSource
1 Claude Fable 5 45.1% HF baseline (Build123d) Jun 2026 Official leaderboard
2 GPT-5.5 Pro 38.7% HF baseline (Build123d) Apr 2026 Official leaderboard
3 Claude Opus 4.7 36.9% HF baseline (Build123d) Apr 2026 Official leaderboard
4 GPT-5.5 36% HF baseline (Build123d) Apr 2026 Official leaderboard
5 Claude Opus 4.8 34.5% HF baseline (Build123d) May 2026 Official leaderboard
6 Gemini 3.1 Pro 31.1% HF baseline (Build123d) Feb 2026 Official leaderboard
7 Claude Opus 4.6 30.9% HF baseline (Build123d) Feb 2026 Official leaderboard
8 Claude Sonnet 4.6 28.3% HF baseline (Build123d) Feb 2026 Official leaderboard
9 Gemini 3.1 Flash-Lite 24.9% HF baseline (Build123d) Mar 2026 Official leaderboard
10 GLM-4.6V 18.3% HF baseline (Build123d) Dec 2025 Official leaderboard
11 Qwen3-VL-235B-A22B-Instruct 16.8% HF baseline (Build123d) Sep 2025 Official leaderboard

Labs report scores for 2 models in their cards. We plot every source in the charts below.

2 models have held number one since its release in May 2026

Lab claims count from the day the card came out, independent runs from the model’s release.

OpenAI Anthropic
Own leaderboard Not tested since Jun 2026
Claude Fable 5 45.1%

Scores over time

All the scores we have, one dot each, so a model tested at several settings appears more than once.

On the benchmark’s own leaderboard Reported by the lab in its model card Best so far
0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Fable 5 GPT-5.5 Pro Gemini 3.1 Pro GLM-4.6V 0% 20% 40% 60% 80% 100% 2026 Near the ceiling (85%) Claude Fable 5 GPT-5.5 Pro

Labs report it only in system cards

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark. Some system cards do, below.

2 system cards report it, from 1 lab

Besides launch posts, we read the system cards, model cards and technical reports labs publish with a release. They report far more, including safety and frontier-risk tests.

DateDocumentScoreAs the document writes itSection
21 Sep 2026 Grok 4.7 model card 44.4 CADGenBench generation split, reward, high, run by Mecado (4.2) Capabilities
12 Aug 2026 Grok 4.6 model card 40.9 CADGenBench generation split, reward, high, run by Mecado (4.4) Capabilities

More in Engineering and CAD

All 18 in Engineering and CAD
  • BenchCAD In use Programmatic CAD: 17,900 execution-verified CadQuery programs over 106 industrial part families. Four matched tasks scored on geometry. GPT-6 Astra 95.9%
  • EEBench In use 13 electrical-engineering circuit and PCB design tasks graded by deterministic build checks, simulation and BOM constraints. No LLM judge. Claude Opus 5.5 75%
  • Parametric CAD Bench In use Agents author editable parametric FreeCAD part designs, graded against held-back references for geometry and spec consistency. Claude Opus 5.5 61%
  • 3DCodeBench Only in system cards Agentic procedural 3D modeling via code: author engine-ready 3D assets through software APIs. Graded for executability and shape fidelity. Grok 4.6 54%
  • PartBench Only in system cards Parametric mechanical part design and CAD-style generation, graded for geometric and manufacturing fidelity. Grok 4.6 58.5%
  • Physical-Engineering's Last Exam Not in model cards Agents produce billable engineering deliverables such as calculation packages, drawings and design reviews in 85 tasks across electrical, civil, structural and mechanical engineering. Automatic graders check constraints, safety limits and the design objective. Claude Opus 5.5 0.49