VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation

VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation, from NVIDIA and the University of Michigan, reframes robot control as “physical orchestration”: instead of treating a vision-language-action model as a fixed executor, VoLoAgent uses a VLM to plan, monitor, interrupt, and recover by steering a VLA/WAM, perception models, and grasp/place primitives as callable tools in one closed loop. To benchmark this setting, the paper introduces RoboVoLo, a high-fidelity benchmark built on RoboLab and NVIDIA Isaac Lab with 4 suites, 15 task categories, and 126 tasks spanning common sense, memory, complex references, and world knowledge. On RoboVoLo, the full system reaches 41.80% overall success, versus 12.57% for a pure VLA, 17.76% for a no-VLA tool-chain, and 34.97% for a VLA-only ablation, with the largest gains coming from monitoring and recovery rather than open-loop action. The paper validates the design on a real Franka FR3 cell: full VoLoAgent achieves 42.9% success on 14 real tasks, compared with 14.3% for π0.5. Failure analysis shows that completion-monitor errors dominate across frontier VLMs, while the orchestrator’s recovery loop substantially improves resilience, raising recovery from 13% to 54% on episodes that do fail. The strongest practical takeaway is that long-horizon manipulation improves when the robot can switch between learned policies and perception-grounded primitives mid-execution, rather than committing to a single monolithic action model.