NTH

HALO: Human-Aligned Data Collection for WhoLe-body MObile Manipulation

AuthorsShuai Yang, Fangyi Xu, Yishu Shen, Kecheng Zheng, Yinghao Xu

AffiliationsThe Hong Kong University of Science and Technology · Robbyant

November 17, 2026 2 min read
Watch on YouTube
The one-line take

HALO helps people teach mobile robots coordinated movement and manipulation more naturally, enabling faster data collection and capable long-horizon tasks.

Key results

2.9
Maximum demonstration collection speedup

× faster than comparison interfaces.

73.3%
Six-stage box task success

Success rate for long-horizon mobile manipulation.

85%
Push Chair success after hand removal

Improved from 55% with deployment-aligned visual processing.

What the paper found

HALO presents a way to collect whole-body mobile-manipulation demonstrations that keeps the operator close to the robot’s hands while allowing natural walking. Operators guide the robot arms through kinesthetic teaching and move a passive, low-resistance platform instead of controlling a powered base. To make these demonstrations suitable for autonomous deployment, HALO aligns platform motion with the robot base’s dynamics and timing, and removes visible human hands using CaRE-Ego, SAM2, and DiffuEraser. The team fine-tunes Physical Intelligence’s π0.5 vision-language-action model and uses asynchronous real-time action chunking to coordinate arm and base movements. Compared with Mobile ALOHA and VR teleoperation, HALO collects demonstrations up to 2.9× faster. Its learned policy succeeds on 73.3% of a six-stage task transporting and repeatedly loading a box, and reaches 75% success on Disassemble Lego. On Push Chair, hand removal in the training videos raises policy success from 55% to 85%, showing the value of aligning demonstration images with deployment observations.

Original abstract

Learning whole-body mobile manipulation requires demonstrations that coordinate bimanual actions with locomotion across extended workspaces. However, existing interfaces trade off direct robot control, perception and action alignment, and natural operator mobility, making such demonstrations difficult to collect. We introduce HALO, a human-aligned data collection system for whole-body mobile manipulation. The system aligns the operator's perception and action space with the robot while preserving direct physical feedback and natural locomotion. Operators guide the robot end-effectors through kinesthetic teaching and move a low-resistance platform without the deployed mobile base while performing bimanual manipulation. This interface enables more natural whole-body demonstrations, but the close human-robot coupling introduces discrepancies between the demonstration and deployment settings. We address these discrepancies with deployment-aligned demonstration processing, which calibrates and temporally aligns platform motion with the deployed base response and masks and inpaints demonstrator hands in recorded observations. Compared with Mobile ALOHA and VR teleoperation, HALO enables up to 2.9x faster demonstration collection while maintaining comparable or higher success rates. Policies trained with HALO perform precise manipulation tasks, including zip-tie fastening, and achieve a 73.3% success rate on a six-stage long-horizon mobile manipulation task involving repeated coordinated base and arm motion. Project page: https://robbyant-research.github.io/Lingbot-ALOHA/

Read the original paper

More in Robotics

Browse all 68 papers →