Dream What I Can: Sparse Visual Planning for Long-Horizon Robot Manipulation

DreamCan tackles a core problem in long-horizon robot manipulation: a video model can imagine a plausible future that a robot cannot actually reach. Instead of planning with language alone or generating dense video trajectories, it selects sparse visual subgoals that balance task progress with reachability. The method learns these two signals from action-free robot videos, uses dynamic programming to annotate useful subgoal sequences, then fine-tunes Wan2.2-TI2V-5B to generate plans. A separate, compact goal-conditioned controller executes the images, while closed-loop replanning lets the planner adapt when reality diverges from its predictions. On LIBERO-Long, the hierarchical system reaches 91.2% success with a 0.1B-parameter policy, outperforming smaller planning baselines and approaching larger models such as π0.5; Lingbot-VA scores higher. On Language Table, DreamCan reaches 86% success on “make a line” and 100% on “all to region,” compared with 56% and 78% for language planning. In real-world tests, it achieves 65% success on box packing with unseen object combinations, and replans every 1.03 seconds on an RTX 5090. The results suggest that choosing executable intermediate images—not simply predicting more frames—can make video-based robot planning more reliable, while supporting test-time search and language-guided plan steering.