Blind Precise Assembly: Proprioceptive Sim-to-Real Peg-in-Hole for Diverse Parts
AuthorsAntonia Bronars, Younghyo Park, Kushal Kedia, Pulkit Agrawal
AffiliationsMassachusetts Institute of Technology
A robot learns to insert, mesh, and thread parts by feeling contact through its own joints, without vision or tactile sensors.
Key results
Worlds used to train policies across randomized assemblies.
Success in 237 zero-shot trials across eight peg designs.
Blind policy success in simulation; the matched visual policy scored 3.9%.
Without supervision, training needed this many times as many environment steps to reach 90% success.
µm error at 20 µm clearance in the mjfit collision benchmark.
What the paper found
This paper shows that a robot can assemble precise parts using only proprioception: its joint positions, velocities, and action history, with no vision, force, or tactile sensing. Using PPO, the researchers trained policies for peg insertion, gear meshing, and nut threading across 12,288 parallel simulation worlds. A temporal convolution encodes a 0.6-second history, while auxiliary contact-state supervision helps learning without giving the policy privileged contact information. The open-source mjfit generator creates exact convex collision geometry for diverse parts, preserving tight clearances; in collision tests, it achieved 0.2 µm worst settled error at 20 µm clearance. The peg policy transferred to a real robot without fine-tuning, succeeding in 73% of 237 trials across eight peg designs, including shapes outside training. In simulation, blind insertion reached 36.2% success on unseen random-convex pegs, versus 3.9% for a matched visual policy. Contact supervision also accelerated learning: without it, peg training took 2.7 times as many environment steps to reach 90% success. The paper additionally benchmarks collision performance with MuJoCo Warp, the GPU simulation system from Google DeepMind and NVIDIA. The approach is a local assembly skill: it begins with the part already touching the fixture and cannot recover from large departures such as a part slipping far from the hole.
Original abstract
Every robot already carries high-resolution contact sensors: its own joints. Legged robots and humanoids exploit them to walk, run, and track whole-body motions without vision, but there the contacts are large and forceful. Is joint sensing also enough for precise, contact-rich assembly, where the robot must find a millimeter-scale opening, mesh gear teeth, or catch a thread through light contact? We study this question by training blind assembly policies that observe only a short history of joint positions, velocities, and their own actions, without visual, force, torque, or tactile sensing. The poses of both the fixture and the part are unknown to the policy, and the grasp is randomized substantially. The policy must therefore assemble the parts through contact, sensed exclusively through its own proprioception. Trained with reinforcement learning entirely in simulation over randomized parts, grasps, and contacts, such policies learn peg insertion, gear meshing, and nut threading. The peg insertion policy transfers zero-shot to the real world and inserts pegs of different shapes and sizes in 73% of trials. Controlled comparisons with matched visual policies in simulation show that blind policies generalize far better to unseen part shapes and sizes, though they can be slower. Training such policies needs thousands of different assemblies, each simulated with its clearance intact, so we also release an open-source generator that builds them procedurally from exact convex pieces for MuJoCo; it is more accurate and faster than convex decomposition or MuJoCo's signed-distance collision. Videos and interactive simulations are at https://younghyopark.me/blind-proprioceptive-assembly/.
Read the original paperMore in Robotics
Browse all 56 papers →Dexterity from Supervised Play
Botao He, Yiming Dou, Yiannis Aloimonos, Jitendra Malik, Akash Sharma, Ruoshi Liu
A robot learns to play from brief human guidance, then uses its own experience to practice and master dexterous manipulation.
Dream What I Can: Sparse Visual Planning for Long-Horizon Robot Manipulation
Chongkai Gao, Wenlong Huang, Haonan Chen, Lin Shao, Yilun Du, Jiajun Wu, Ruohan Zhang, Li Fei-Fei
DreamCan turns imagined video futures into sparse, achievable visual goals to help robots complete long and complex manipulation tasks.
Dex-One2Many: Learning Dexterous Manipulation from a Single Human Demonstration
Jusuk Lee, Sungha Kim, Yeonsoo Park, Jonguk Cheon, Yoonkyo Jung, Yongjun You, H. Jin Kim, Jia-Bin Huang, Furong Huang, Youngseok Jang, Seungjae Lee
A single human video is transformed into scene-graph guidance that helps a robot hand learn dexterous skills and adapt to situations it never saw.