VA-TDM: A Unified View for Improving Few-Step Diffusion Models by Velocity Adversarial Distillation
AuthorsYihong Luo, Tianyu Pang, Tianyang Hu
AffiliationsHong Kong University of Science and Technology · The Chinese University of Hong Kong, Shenzhen Summarized Prompt: A woman wearing headphones journeys through surreal worlds, ending at a mountain sunrise with the text "hear beyond reality"
VA-TDM uses adversarial feedback on diffusion velocities to produce high-quality images and videos with just a few generation steps.
Key results
VA-TDM generation budget for the reported few-step results.
VA-TDM score, compared with TDM's 0.61.
VA-TDM score, compared with 0.64 for the released four-step model.
VA-TDM score, compared with 0.47 for the released four-step model.
Model scale used to demonstrate video-generation distillation.
Human preference for VA-TDM at the same four-step budget.
What the paper found
VA-TDM addresses a central problem in few-step diffusion: student-score estimates can fail to represent the distributions produced during intermediate denoising. Its underlying Velocity Adversarial Distillation framework aligns teacher and student velocity fields using a learned vector-valued critic. It combines A-form rectified velocities, which provide conditional-moment supervision, with B-form Gaussian velocities, which encode distributional information; together, these connect moment matching, score distillation, and Fisher-based matching. VA-TDM applies this dual-form supervision along partial denoising transitions in the student’s own trajectory, using transition-aware online field estimation and a shared backbone, without adding inference cost. At 4 NFE, it raises SD3.5-Medium’s GenEval score from TDM’s 0.61 to 0.67. On FLUX.2-Klein-Base 4B, it improves OCR similarity from 0.64 to 0.69 and LPIPS diversity from 0.47 to 0.56. The method also scales to 33B MiniMax-H3: in human comparisons, its four-step video generation is preferred over Fast-H3 61.33% of the time, and it receives 45.53% preference against the 100-NFE MiniMax-H3 teacher, compared with 24.67% for Fast-H3.
Original abstract
Few-step distillation offers efficient generation with diffusion and flow models, but preserving quality with only a few sampling steps remains challenging. Distribution-matching methods via score distillation provide a strong foundation, yet they rely on accurately tracking the student's changing distribution. At intermediate generation steps in few-step sampling, student-score estimation can be difficult, and commonly used estimators may describe a different distribution from the one used to update the student. Flow matching allows a more flexible approach to supervision through velocity fields constructed from the student's sampling process. Building on this perspective, we introduce Velocity Adversarial Distillation (VAD), which uses a learned critic to compare student and teacher fields and turn their disagreement into feedback for learning. VAD unifies score distillation, Fisher-based matching, and moment matching, also explaining existing heuristic MMD updates and delivering new training objectives. We further develop VA-TDM, which follows trajectory distribution matching: providing supervision for partial denoising along the student trajectory. Beyond this, VA-TDM combines transition-aware online velocity learning and dual-form VAD supervision across intermediate student transitions. Extensive results show that VA-TDM is powerful for learning few-step generators, achieving state-of-the-art four-step performance in both text-to-image and text-to-video generation across various backbones and scaling effectively to 33B MiniMax-H3. In particular, our VA-TDM-H3 notably outperforms the few-step Fast-H3 and is closer to the original many-step H3 according to human preference.
Read the original paperMore in Diffusion Models
Browse all 82 papers →FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning
Haitao Lin, Minsi Ren, Yang Li, Xingtong Ge, Zheyuan Hu, Peiyan Hu, Tailin Wu
FewStepRL adapts multiple reinforcement-learning fine-tuning methods to make reward-driven improvements to fast, few-step diffusion models.
JVT: Just Video Transformer makes pixel-space video diffusion simple
Jianjun Zhou, Hua Geng, Haoyu Guo
JVT shows that a straightforward Transformer can generate video directly in pixels—and even produce matching depth-like geometry alongside it.
Restoring Diversity in Distilled Image and Video Generators by Probing Activations
Furkan Horoz, Ozgur Kara, Kaan Oktay, Adil Kaan Akan, Atakan Kara, Anand Bhattad, James Matthew Rehg
The paper restores variety in fast distilled image and video generators by finding and correcting a collapse in their internal activation directions.