NTH

Future Representation Distillation for Imagination-free World Action Models

AuthorsJaehyeong Jo, Sangwon Jang, Sihyun Yu, Seonghyeon Ye, Yekyung Nah, Hojung Jung, Kimin Lee, Sung Ju Hwang

AffiliationsKAIST

November 9, 2026 2 min read
Watch on YouTube
The one-line take

DreamFree teaches a robot to use a video model’s learned sense of the future without having to generate that future frame by frame.

Key results

14B
Video backbone size

Wan2.2 backbone used for the reported real-time policy.

11
Closed-loop control rate

Hz achieved with the 14B backbone on an NVIDIA H200.

17.8
Speedup over DreamZero

Inference speedup without future-video generation.

71.7%
RoboCasa-Kitchen success

Average success rate using the 14B backbone.

64%
Real-world success

Average rate across 10 DROID tasks, versus 56% for DreamZero and 16% for π0.5.

What the paper found

DREAMFREE makes a video diffusion model’s knowledge of how scenes evolve useful for robot control without generating future videos at deployment. In a two-stage process, a visual encoder initialized from Wan2.2 learns from the current image and instruction to predict representations produced by a frozen teacher with access to future frames; a structured attention mask prevents the encoder from seeing those future frames. The encoder is trained with both future-representation distillation and the video model’s generative objective, then an action readout and a 1B-parameter action DiT map its features to robot actions. This also lets the visual encoder learn from videos without action labels. With a 14B backbone, DREAMFREE reaches 11Hz closed-loop control on an NVIDIA H200 and runs 17.8 times faster than DreamZero. It achieves 71.7% success on RoboCasa-Kitchen, and in real-world tests its 64% success rate exceeds DreamZero’s 56% and π0.5’s 16%. The results show that distilling predictive features can improve control while avoiding the cost and potential errors of future-video generation; video-only task demonstrations also enable transfer to tasks lacking action annotations, including across robot embodiments.

Original abstract

World Action Models (WAMs) use the predictive knowledge of pretrained video diffusion models for robot control through future video prediction. However, iterative video denoising is costly, and inaccurate visual predictions can compromise action prediction. Our key idea is to make this predictive knowledge available through visual representations computed from the current observation, enabling action prediction without generating future videos. We introduce DreamFree, a WAM that repurposes a pretrained video diffusion model as a predictive visual encoder through representation distillation. The visual encoder learns to predict a frozen video diffusion model's representation using only the current observation. An action module is then trained to predict actions by extracting action-relevant information from the visual encoder. Across real-world and simulated tasks, DreamFree achieves higher success rates than state-of-the-art VLAs and WAMs, while performing real-time closed-loop control at 11Hz with a 14B video diffusion backbone. Experiments show that DreamFree can translate stronger video priors and additional video data into better robot control and enable learning new robot tasks from video demonstrations, including videos from a different embodiment, without action annotation for the new tasks.

Read the original paper

More in World Models

Browse all 59 papers →
01World Model

Duo: A Reciprocal World Model for Full-Duplex Interaction

Hengyuan Cao, Biao Gong, Guoshan Lu, Qiyuan Zhang, Lin Liu, Yiding Tian, Yilei Jiang, Yuxin Ma, Chengzhang Wu, Kun Chen, Yiqian Xie, Ting Liu, Xin Zheng, Xijun Gu, Tianren Ma, Jiancheng Pan, Chi Zhang, Xuzhe Zheng, Siyi Li, Zhenzhong Lan

Duo aims to create an always-on audiovisual assistant that can perceive, reason, and respond continuously, including during interruptions.

Read analysis
02World Model

PanDWM: Panoramic 4D Driving World Models

Jiajun Sun, Zeyu Zhang, Zeyu Ren, Guangfeng Jiang, Yang Zhao, Hao Tang

PanDWM generates a continuous 360-degree driving view and reconstructs it into a fast 4D scene.

Read analysis
03World Model

WorldRuler: Interactive World Modeling under Configurable Rules

Mingliang Zhai, Chuanhao Li, Jiaming Tan, Zhen Li, Yuwei Wu, Yunde Jia, Zian Meng, Yuanyang Yin, Kaipeng Zhang

WorldRuler aims to make generated interactive worlds obey customizable rules, even after saving and restoring the world.

Read analysis