Benchmarks / Parametric CAD Bench V3
Parametric CAD Bench V3 (CAD Bench V3)
Built by gNucleus AI · released 21 Sep 2026
100 FreeCAD modeling tasks from gNucleus AI: 30 parts to create from a text spec, 30 parts to create and then edit, and 40 parts to build from an engineering drawing. A coding agent works in a terminal container, and a deterministic geometry scorer grades the FreeCAD model that it produces. V3 (September 2026) is a new series, separate from the frozen V1 and V2 boards.
Frontier
61%
Overall score
Claude Opus 5.5 (max) with Claude Code 2.1.280 · harness: Claude Code 2.1.280
24 Sep 2026 · Source: gNucleus AI (benchmark maintainers)
Added 2026-09-24 (maintainers' post "Claude Opus 5.5 Takes the Lead"). Site shows 61.03% ± 7.53; we computed the interval from the ±. Create 50.87, create-and-edit 39.69, image-to-CAD 84.66 (the board's highest). Scored 99/100, 9 perfect tasks, $573.94 for the cohort. One final output-limit error counts as zero; Claude Code fell back to Opus 5 on two zero-scoring tasks, with no score benefit. The same model with mini-swe-agent scores 57.96% (8 perfect). The maintainers say the confidence intervals of the top four rows overlap, so the order is not statistically decisive.
The best result is at 60–90% of the ceiling, or at or above human parity.
What it measures
Mean continuous task reward over the 100 tasks, shown on a 0 to 100 scale. The verifier (gnucleus-freecad-validator 0.6.0, FreeCAD 1.1.0) compares the produced model with a held-back reference model and the spec: an oriented-bounding-box size gate, per-component comparisons, geometry weights, and spatial alignment. A failed or unscored trial counts as zero. Each official row is one fresh run with one attempt per task, no network except the model provider, and a public Harbor job that anyone can inspect. A 95% confidence interval is computed over the 100 per-task rewards.
Overall score: Mean continuous task reward across the 100 tasks, on a 0 to 100 scale; failed or unscored trials count as zero. Higher is better.
Status compares the frontier with a ceiling of 100%.
Facts
- Grain
- Task level: bounded tasks with a clear spec
- Environment
- cli
- Grading
- automated-tests
- Tasks
- 100
- Contamination
- Reference geometry stays in a separate verifier image. Agents run with no network except the model provider. In V2 the maintainers saw agents try to find or download the reference CAD or the grader, which is one reason for the stricter V3 isolation.
- Reuse
- The V3 task suite is distributed through Harbor Hub (gnucleus-ai/cad-bench@v3); we found no license text for it. The reference CAD models are held back in a separate verifier environment. The verifier (freecad-validator) is Apache 2.0. The V1 and V2 results archives on Hugging Face are Apache 2.0. (cite-only)
Limits to keep in mind
- The top four rows (61.03%, 57.96%, 56.87%, 56.75%) have overlapping 95% confidence intervals, so the maintainers say their order is not statistically decisive on 100 tasks. Source
- Every row is a model plus an agent harness. The same Gemini 3.8 Flash model scored 36.19% with mini-swe-agent and 27.56% with Antigravity, so rows are not clean model rankings. Source
- All rows so far are the maintainers' own runs of frontier vendors' models. At the V3 release, six create-and-edit tasks scored zero for every row. Source
- gNucleus AI sells CAD generation models and agents, so the maintainer has a commercial stake in the field it benchmarks. Source
Recorded results
A selection that shows the frontier over time. The full leaderboard is at the source.
| System | Overall score | Perfect tasks | Date | Source |
|---|---|---|---|---|
| Claude Opus 5.5 (max) with Claude Code 2.1.280 · frontier harness: Claude Code 2.1.280 | 61% | 9% | 24 Sep 2026 | gNucleus AI · primary |
| Claude Opus 5 (max) with Claude Code 2.1.270 harness: Claude Code 2.1.270 | 50.9% | 6% | 21 Sep 2026 | gNucleus AI · primary |
| Claude Fable 5.1 (max) with Claude Code 2.1.270 harness: Claude Code 2.1.270 | 56.8% | 7% | 21 Sep 2026 | gNucleus AI · primary |
| GPT-6 Astra (max) with Codex 0.154.0 harness: Codex 0.154.0 | 56.9% | 5% | 21 Sep 2026 | gNucleus AI · primary |
Timeline
Where it sits in the atlas
Work ladder: Direct evidence for Architecture and engineering. On the work ladder it counts as 9% of tasks fully completed (Perfect tasks). How the ladder works
Work it measures (O*NET work activities): Create visual designs or displays; Design materials or devices; Evaluate designs, specifications, or other technical data.
Go to the source
- Website cadbench.ai
- Full leaderboard cadbench.ai
- Code github.com
- Announcement cadbench.ai
- Dataset hub.harborframework.com
Last checked 10 Oct 2026 against 4 primary sources. See an error? Tell us.
How to cite
Credit the original work first: Parametric CAD Bench V3 by gNucleus AI.
Then, if you used this page:
Can Agents Work. "Parametric CAD Bench V3: frontier results and sources." https://canagentswork.com/benchmarks/parametric-cad-bench-v3/ (accessed 2026-10-11). CC BY 4.0.@misc{caw-parametric-cad-bench-v3,
title = {{Parametric CAD Bench V3: frontier results and sources}},
author = {{Can Agents Work}},
year = {2026},
howpublished = {\url{https://canagentswork.com/benchmarks/parametric-cad-bench-v3/}},
note = {Accessed 2026-10-11. CC BY 4.0}
}