DrawScore: Evaluating Generated CAD Models against Engineering Drawings
AuthorsHaorui Chen, Shiteng Lu, Zhanyu Guo, Zaixi Li, Yunhe Ni, Yaosong Du, Yichen Xiang, Xiande Huang, Rui Huang
AffiliationsUniversity of Electronic Science and Technology of China · De Artificial Intelligence Technology Co., Ltd · University of Hong Kong
DrawScore checks generated CAD models through engineering-style views, making small but manufacturing-critical mistakes harder to overlook.
Key results
The contribution of a local error decays linearly with its size.
Outputs from six agent configurations used in the metric comparison.
Share of outputs with IoU ≥ 0.95 that scored below 0.95 on DrawScore.
Share of outputs with DrawScore ≥ 0.95 that scored below 0.95 on IoU.
Increased from 75.2% for the supervised fine-tuning model.
What the paper found
DrawScore targets a blind spot in common CAD metrics: a misplaced hole or slot can be critical to manufacturing yet barely change volumetric IoU or surface-based scores. Its analysis shows that local-error contributions shrink according to the metric’s integration dimension and penalty; DrawScore reduces this decay to linear, order 1. It projects each model into front, top, and right views, compares visible and hidden curves using a saturating Gaussian penalty with tolerance σ = 0.005L, and takes the lowest view score. Across 1,188 outputs from six CAD-agent configurations, 18.9% of models with IoU ≥ 0.95 scored below 0.95 on DrawScore, while only 1.2% of models with DrawScore ≥ 0.95 had IoU below 0.95. As a reinforcement-learning reward for Qwen3.5-4B, DrawScore raised the valid-output share from 75.2% to 89.5% and improved every reported geometric metric over supervised fine-tuning. At inference time, it also selected better candidates by comparing their projections with the input drawing, without needing 3D ground truth or a separate reward model.
Original abstract
Automatic CAD generation is commonly evaluated with volumetric IoU, Chamfer Distance, and surface F-score, which barely penalize local errors that occupy only a small fraction of a part's volume or surface area, such as a misplaced hole or a slot on the wrong face, even though such errors are critical to manufacturing. We formalize these metrics in a unified integral form and show that the weight of a local error decays as the $(k+q)$-th power of its size, where $k$ is the dimension of the integration domain and $q$ is determined by the penalty function. Guided by this analysis, we propose DrawScore, which projects both models into three orthographic views with visible and hidden lines, as in engineering drawings, and applies a saturating penalty along the projected curves, so that the weight of a local error decays only linearly. Controlled experiments confirm the predicted decay orders, and on outputs of existing CAD generation agents, DrawScore exposes errors in outputs that IoU rates as nearly perfect. DrawScore further improves generation as a reinforcement learning reward and, for CAD generation from engineering drawings, serves as a test-time verifier without 3D ground truth or an additional reward model. Our code is available at https://github.com/UESTC-Simon/DrawScore.
Read the original paperMore in AI Benchmarks
Browse all 60 papers →Accelerate Bench: Evaluating Agents for Application-Level Program Optimization
Bowen Liu, Yuhan Li, Nuo Chen, Weile Luo, Miao Peng, Qintong Zhang, Qifan Zhang, Jia Li, Ziqi Gao, Lydell Huang
A new benchmark tests whether coding agents can make whole scientific programs faster without breaking them.
PReMind: When Personal Memory Meets Visual Hazards
Yeongtak Oh, Donghyeok Lee, Joonhee Oh, Jaewon Jeong, Sungroh Yoon
PReMind tests and trains visual assistants to connect what they see with the personal context needed to give safer advice.
AutoVend: Benchmarking LLM Agents on Long-Horizon Autonomous Vending Operations
Qiming Shi, Yikun Li, Beidi Luan, Rui Sun, Sinuo Wang, Delin Wang, Yan Ma, Jing Li, Daxin Jiang, Zuo Bai
AutoVend tests whether LLM agents can run vending-machine businesses profitably over six months—and finds that most struggle to beat a simple inventory baseline.