Learning Heterogeneous Tactile Representation for General Dexterous Manipulation
AuthorsBing Han, Yichuan Peng, Haoyu Dong, Yuanmeng Hu, Yongyi Jia, Fangjing Wang, Kecheng Zheng, Yiming Zhang, Yanhong Zeng, Xiaoqing Zhang, Chen Song, Rundong Zou, Anqi Dong, Yuan Guan, Ka Leong Cheng, Xing Zhu, Yujun Shen, Nan Xue, Wei Ning
LingBot-Touch aims to help robots manipulate objects more dexterously by learning tactile representations that work across different sensor types.
Key results
Scale of the heterogeneous tactile corpus.
Episodes spanning the three sensor types.
Total frames in the tactile corpus.
Percentage-point gain averaged over 26 real-robot tasks.
Average across 12 dexterous tasks using LingBot-Touch features.
Parameter count of the compact tactile encoder.
What the paper found
LingBot-Touch tackles a core obstacle in robotic touch: piezoresistive, magnetic, and visuotactile sensors record contact in incompatible formats. Its pretraining corpus contains 4983 hours, 464431 episodes, and 1.33B frames. The model maps taxels into a shared canonical fingertip space, uses sensor-specific tokenizers and a shared Haptic Transformer, and learns with Tactile-JEPA, which predicts masked tactile representations in latent space rather than reconstructing mostly empty signals. Layout augmentation helps the encoder transfer to unfamiliar sensor geometries. Frozen features reveal contact force, location, area, and motion; on the unseen XHand1 sensor, tactile-to-video retrieval reaches 47.1% top-5 recall without sensor-specific adaptation. The same tokens plug into policies including Diffusion Policy, π0.5, and LingBot-VLA: across 26 real-robot tasks, LingBot-VLA’s average success rises from 48.3% to 73.1%, a gain of 24.8 percentage points. In simulation, LingBot-Touch features achieve 69.0% average success across 12 dexterous tasks, compared with 45.6% using a task-trained tactile CNN and 21.4% without touch. The Tiny encoder has 5.46M parameters, making the approach a compact way to give existing manipulation policies a reusable representation of contact.
Original abstract
Tactile sensing is an indispensable modality for contact-rich dexterous manipulation, a capability that hints at the ultimate frontiers of embodied intelligence. However, commercially available tactile sensors are still at a relatively early stage of development, with each type of sensor offering irreplaceable strengths alongside its own drawbacks. A key challenge is thus to free tactile understanding from sensor-specific physics, so that embodied models can effectively leverage heterogeneous tactile data. To this end, we develop LingBot-Touch, a tactile foundation model characterized by the following properties. (i) Our model yields a unified representation across different tactile sensors (i.e., piezoresistive, magnetic, and visuotactile sensors), and is easily extensible to new sensor types. (ii) The resulting tactile tokens are plug-and-play: they can be seamlessly integrated into policy models across paradigms, from diffusion policy to vision-language-action (VLA) models such as $π_{0.5}$ and LingBot-VLA, consistently enhancing manipulation dexterity. (iii) Extensive experiments, spanning simulation and real-robot deployments across multiple hardware platforms, demonstrate that our tactile representation substantially benefits complex manipulation tasks while maintaining strong computational efficiency. We believe the findings in this work demonstrate the potential of leveraging cross-sensor data to scale tactile learning, and offer encouraging insights for the design of future hardware and embodied models.
Read the original paperMore in Embodied AI
Browse all 69 papers →Demonstration of MANA: Multi-perspective Alignment of NPC Agents
Amir Tahmasbi, Keshav Singh, Shaheer Raza, Owen Mei, Jason Wu
MANA lets game developers step into an NPC’s shoes and demonstrate how it should behave, then lets the character act on its own.
Kimodo-Policy: Learning Kinematic Action Models for Humanoid Loco-Manipulation
Yunheng Wang, Jiaxi Zhang, Wei Lu, Yuetong Fang, Taowen Wang, Yixiao Feng, Shuning Zhang, Junzhe Xu, Yichi Wang, Zizhao Yuan, Kailun Huang, Lusong Li, Mingqian Zhou, Zecui Zeng, Renjing Xu
Kimodo-Policy turns a pretrained whole-body motion generator into a control policy for humanoid robots that walk and manipulate objects.
One Policy for 100 Household Tasks: Our Solution for the 2026 BEHAVIOR Challenge
Ilia Larchenko, Gleb Zarin, Atindra Nair, Keerti Manney, Vitaly Kleban, Mirko Kiefer, Anjana Yodaiken
A single vision-language-action policy tackles 100 simulated household tasks, with a coding agent stepping in for some of the hardest ones.