SuperNav: An Agentic Navigation System for Any Task in Any Scene
AuthorsJinkai Zhang, Jingyi Xu, Yuanhong Yu, Jiarui Guo, Ruizhen Hu, Hujun Bao, Xiaowei Zhou, Sida Peng
AffiliationsZhejiang University · Shenzhen University
SuperNav turns a general-purpose multimodal model into a versatile robot navigator by letting it reason about tasks and scenes while specialized tools handle movement.
Key results
SuperNav with the Geo-based Executor; strongest baseline, UniNaVid, scored 34.00%.
SuperNav with the Geo-based Executor on Demand-Bench tasks; strongest baseline scored 37.50%.
SuperNav’s result on ordered multi-object navigation.
SuperNav with the Geo-based Executor at a 1 m success threshold.
What the paper found
SuperNav tackles open-ended robot navigation without navigation-specific fine-tuning of its multimodal language model. Instead, an agent harness lets OpenAI’s GPT-5.6 Terra interpret requests, inspect scenes, track task progress, and call motion tools; reusable Skills guide search, verification, and recovery. Its central interface lets the agent choose a destination as a normalized image point in one of four directional RGB views, then use execution feedback to revise its plan. That interface supports either a geometry-based executor using depth and navigation meshes or a learned, NoMaD-inspired RGB controller. In evaluation, SuperNav reached 78.00% success on single-object tasks, versus 34.00% for the strongest baseline, and 59.50% on Demand-Bench tasks, versus 37.50%. It achieved 34.00% on ordered multi-object navigation and 73.33% on HM3D-OVON at a 1 m success threshold. The system was also deployed on a Unitree Go2 quadruped. The results suggest that combining a general-purpose model with tools and task-state management can transfer across varied navigation requests, though semantic judgments still depend on the model and long tasks remain challenging. The agent runtime uses OpenAI Codex.
Original abstract
General-purpose service robots need navigation systems that can handle diverse human requests in unfamiliar environments, combining task generality with scene generality. Some existing methods fine-tune multimodal large language models (MLLMs) to predict navigation actions, making their behavior dependent on the coverage of navigation training data and potentially limiting generalization to new requests and environments. Our key insight is to let the MLLM focus on interpreting requests, understanding scenes, and making decisions while preserving its general-purpose capabilities and delegating motion execution to navigation tools. To realize this idea, we introduce SuperNav, which equips a pretrained MLLM with a specialized agent harness without navigation-specific fine-tuning of the MLLM. Our harness supports these decisions with Navigation Skills, agent-oriented Tools for physical interaction, and task-progress and context management. A unified visual-point interface connects decision-making to motion by allowing the model to specify destinations directly in images and revise its decisions from execution feedback. Together, these components support sustained navigation across different task requirements and environments. SuperNav outperforms four evaluated baselines on instance-level, multi-object, and demand-driven tasks. Category-level evaluation on HM3D and deployment on a real quadruped robot further demonstrate its applicability across environments. Project Page: https://zju3dv.github.io/SuperNav/
Read the original paperMore in Embodied AI
Browse all 51 papers →In-context Robot Learning Made Simple: A Democratized Recipe for Manipulation Tasks
Minxing Li, Minghao Han, Weizhi Zhao, Hanwen Wang, Xiangshuo Liu, Shuyao Shang, Jingxiang Zhou, Mingchao Sun, Hongyu Pan, Mu Xu, Yu Liu, Lue Fan, Zhaoxiang Zhang
SimpleICL presents an accessible recipe for teaching robots manipulation tasks from visual demonstrations without massive pretraining or specialized infrastructure.
NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime
Gengze Zhou, Yicong Hong, Jiazhao Zhang, Xunyi Zhao, Jian Zhou, Zixing Lei, Zun Wang, Chongyang Zhao, Xionghui Chen, Stephen Gould, Anton van den Hengel, Qi Wu
NavGPT-3 pairs a reasoning language model with a fast action policy so a robot can navigate complex routes at human-level performance.
GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
Qize Yu, Lianrui Fan, Boyu Chen, Jiaqi Liang, Xini Ding, Yue Chen, Zetian Song, Yuran Wang, Yi Zou, Kaixuan Wang, Tianxing Chen, Wenxuan Song, Bohan Zhou, Mingleyang Li, Siqiao Huang, Yuqi Ye, Caigao Jiang, Wei Wei, Ruihai Wu, Hang Zhang, Yixiao Ge, Shuchang Zhou, Shilong Liu, Xianming Liu, Ping Luo, Shiyu Huang
GroundingPI argues that fast, precise visual grounding should be the perceptual foundation for capable robots and autonomous vehicles.