Rolling-WAM: World Action Models with Rolling Imagination

Rolling-WAM is a World Action Model that jointly denoises future video and robot actions, but distributes that computation across replanning cycles instead of restarting from Gaussian noise each time. It maintains a sliding window of aligned video-action chunks at staggered noise levels: the imminent chunk is fully denoised for execution, while farther chunks are progressively refined and carried forward after each new camera observation. With a default window of 5 chunks, 16 actions per chunk, and only 2 denoising steps per steady-state update, the model preserves visual imagination across an 80-action horizon. Its Mixture-of-Transformers design combines a pretrained Wan2.2-TI2V-5B video expert with an approximately 1B-parameter action transformer, using flow matching and masked cross-chunk attention. On an NVIDIA A100, Rolling-WAM reaches 215 ms replanning latency versus 978 ms for Joint-WAM, a 4.5x speedup, while retaining future-video prediction. It achieves 98.1% average success on LIBERO, 93.3% on RoboTwin 2.0, and 85.0% across three real-world Unitree G1 humanoid tasks, outperforming Joint-WAM’s 78.3% real-world average and Fast-WAM’s 75.0%. The system was evaluated alongside models such as π0.5 and GR00T N1.7, with practical relevance to low-latency physical-AI systems developed by companies including Toyota Research Institute.