JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion

The JD-affiliated Joy Future Academy team presents JoyAI-Video-Edit, a 16B-parameter autoregressive diffusion system for open-ended video editing that generates causally, without future frames or a fixed duration. Its architecture combines an MLLM condition encoder, causal video VAE, and multimodal diffusion transformer, using chunk-wise attention, bounded sliding-window KV caching, and a global first-chunk sink to keep memory and per-chunk computation constant. Two key training techniques address autoregressive failure: Source-Anchored Distribution Matching Distillation, or SA-DMD, compresses diffusion into a two-step generator while anchoring each prediction to the aligned source chunk, and Long-Horizon Autoregressive Distillation trains on extended rollouts to reduce accumulated temporal drift. With FP8 quantization, operator fusion, and optimized VAE execution, the system reaches 30.19 FPS for end-to-end 720p editing on a single Nvidia B200 GPU. On the short-video OpenVE-Bench, judged by a Gemini multimodal evaluator, it scores 3.60 overall and outperforms streaming baselines while approaching offline systems such as Runway Aleph and Kuaishou’s Kling-3.0 Omni. The authors also introduce LongV2VBench, containing 229 one-minute editing tasks; JoyAI-Video-Edit scores 3.30 overall and ranks first across all five categories, demonstrating sustained quality over long streams rather than only short clips.