S2PD: Serial-to-Parallel Diffusion for Physically and Logically Consistent Video Generation
AuthorsJeffrey Hu, Daniel Olmeda Reino, Ayush Tewari
AffiliationsUniversity of Cambridge · Toyota Motor Europe
S2PD first reasons through video events step by step, then refines them in parallel to make generated videos more physically and logically consistent without the full cost of autoregressive sampling.
Key results
Size of the pixel-space diffusion transformer trained from scratch.
Invalid transitions per rollout, compared with 9.4 for the bidirectional baseline.
Invalid transitions per rollout, compared with 47.9 for the bidirectional baseline.
S2PD score, compared with 698.7 for the bidirectional baseline.
Seconds for S2PD, compared with 13.8 seconds for cDF.
What the paper found
S2PD addresses a weakness in bidirectional video diffusion: even with effectively unlimited procedural training data, models can produce physically impossible or rule-breaking events. It first denoises video blocks autoregressively at high noise, then switches to joint parallel refinement at a transition noise level of 0.6, using serial computation to establish coherent event structure and parallel computation to stabilize appearance. The method is tested with a 132.7M-parameter pixel-space DiT-B and a pretrained Wan-5B model adapted with LoRA. On Conway’s Game of Life, invalid transitions fall from 9.4 per rollout for the bidirectional baseline to 0.0 with S2PD; on 2048, they fall from 47.9 to 0.9. On Rubik’s Cube Real, S2PD achieves a CD-FVD score of 64.4, compared with 698.7 for the bidirectional baseline. For Kubric MOVi-A, sampling takes 6.8 seconds, versus 13.8 seconds for the serial cDF baseline, while parallel refinement also reduces flicker. The paper positions this approach alongside interactive world models such as Google DeepMind’s Genie 3 and NVIDIA’s Cosmos 3, while emphasizing that S2PD does not require dense action conditioning. It also discloses using OpenAI Codex and Anthropic Claude for coding, planning, and editing.
Original abstract
Bidirectional video diffusion models denoise entire videos in parallel, yet when trained on effectively unlimited in-distribution data from procedural generators, continue to violate physical laws and simple symbolic rules. We introduce Serial-to-Parallel Diffusion (S2PD), which performs autoregressive diffusion at high noise before switching to parallel diffusion at low noise. The autoregressive phase provides the serial computation needed to coordinate interdependent events and produce valid state transitions while the parallel phase jointly refines the entire video and reduces sampling time relative to fully serial generation. We implement S2PD with two architectures: a pixel-space diffusion transformer trained from scratch and a pretrained video model adapted through LoRA fine-tuning with causal attention. Across games, physical simulations, and real video, S2PD follows rules more reliably than matched bidirectional baselines and generates videos with greater temporal stability and sampling efficiency than other serial methods.
Read the original paperMore in Diffusion Models
Browse all 64 papers →DuoMatching: Joint-Marginal Distribution Matching for Few-Step Video Generation
Jiahao Zhan, Yan Wang, Yongrui Ma, Qunliang Xing, Ruchang Yao, Runtao Liu, Shijie Zhao, Tianfan Xue
DuoMatching improves few-step video generation by combining video-level motion supervision with image-level visual and semantic guidance.
In-Distribution Forcing for Long Video Generation at Test Time
Jeongwoo Shin, Youngyoon Choi, Sangwoo Jo, Hyunmog Kim, Sungjoon Choi, Joonseok Lee, Jaewoong Choi, Jaemoo Choi
ID-Forcing stabilizes long video generation by ensuring that cached attention states remain consistent with the model’s training distribution.
ALoDLM: Adaptively Looped Diffusion Language Models
Liancheng Fang, Zhuowei Li, Youngeun Kim, Tianchen Zhao, Rajat Koner, Jiaye Wu, Linghan Xu, Xuanbai Chen, Xiang Xu, Zheng Zhang, Jakub Zablocki, Nishant Sankaran, Yifan Xing
ALoDLM makes diffusion language models smarter and faster by spending extra computation only on tokens that are difficult to predict.