SimRepairBench
Not in model cards
Agents find and repair modeling errors in 3D OpenUSD assets for robot simulation, from 25 damaged objects in NVIDIA's SimReady collection, with partial credit from verifiers.
Release 0.4.13 on Harbor Hub holds the 25 CASTAIGNE25 tasks. Each task gives a damaged model, a repair brief and an image, and a separate verifier scores the repair from 0 to 1. Source objects are CC BY 4.0, the code is MIT.
- Level
- Expert Specialists with years of training, often over hours of work
- Scored
- Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
- Answers
- Agent task Many steps in a terminal, browser, desktop, app, game or robot.
No scores here yet
No lab, leaderboard or Epoch AI run has reported a score for SimRepairBench yet.
No lab model card reports it yet
We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.
More in Engineering and CAD
- BenchCAD In use Programmatic CAD: 17,900 execution-verified CadQuery programs over 106 industrial part families. Four matched tasks scored on geometry. GPT-6 Astra 95.9%
- EEBench In use 13 electrical-engineering circuit and PCB design tasks graded by deterministic build checks, simulation and BOM constraints. No LLM judge. Claude Opus 5.5 75%
- Parametric CAD Bench In use Agents author editable parametric FreeCAD part designs, graded against held-back references for geometry and spec consistency. Claude Opus 5.5 61%
- CADGenBench Only in system cards Agents construct CAD models from design prompts. The generation split is graded for executability and geometric correctness. Claude Fable 5 45.1%
- 3DCodeBench Only in system cards Agentic procedural 3D modeling via code: author engine-ready 3D assets through software APIs. Graded for executability and shape fidelity. Grok 4.6 54%
- PartBench Only in system cards Parametric mechanical part design and CAD-style generation, graded for geometric and manufacturing fidelity. Grok 4.6 58.5%