One Demonstration, Many Objects: Generalizing Manipulation via Local Contact Geometry

DemoMimic learns dexterous manipulation from a single human demonstration and transfers it across unseen objects by exploiting local contact geometry rather than matching complete object shape. The framework combines residual reinforcement learning with 2 contact-centric rewards: Alignment Reward, which aligns hand and object surface normals, and Sustained Contact Reward, which favors uninterrupted contact streaks. A high-level Diffusion Policy predicts a wrist trajectory from one egocentric RGB image, while a low-level Diffusion Policy uses wrist-mounted depth, proprioception, and the trajectory for closed-loop control without explicit object-pose estimation. Demonstrations come from the ARCTIC dataset or manually captured motion using Meta Quest tracking; simulation imagery is diversified with NVIDIA’s Cosmos-Transfer-2.5, and deployment depth is produced by FoundationStereo. On 16 objects spanning 4 tasks and 2 robot-hand embodiments, the system reaches 71% real-world success, with 76% on the Sharpa hand and 65% on the Tesollo hand. Generalization fails mainly when local contact geometry changes: the robot-hand box reaches only 39% on the open-the-box task. Ablations show that removing either contact reward increases hardware failures, despite near-ceiling simulation performance, demonstrating that precise alignment and persistent forceful contact are central to sim-to-real transfer.