VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection
AuthorsKe Sun, Shuwei Li, Haiyang Mei, Mike Zheng Shou
AffiliationsShow Lab, National University of Singapore
A periodic trace left by video-generation VAEs could help spot AI-made videos even when their visible artifacts are gone.
Key results
Variants spanning 31 video generators in the rhythm analysis.
Training-free VAE Rhythm score.
Zero-shot score advantage.
Largest AUROC gain among 19 tested detectors.
Detector trained with benchmark data and VAE pairs.
Generator-free CaDet variant trained without generated videos.
What the paper found
This paper introduces VAE Rhythm, a hidden periodic signal caused by temporal frame grouping in causal video VAEs. Instead of relying on visible artifacts, its training-free phase scan reconstructs a clip at successive start offsets with a frozen Wan2.1 4× VAE, subtracts each frame’s baseline reconstruction error, and measures the remaining periodicity. The signal appears across 39 variants of 31 generators, including OpenAI’s Sora, Kling 3.0, Seedance 2.0, Grok Imagine, and Veo 3.1. On DF26, the zero-shot rhythm score reaches 76.2% AUROC, beating the strongest retrained baseline by 5.8%. As a plug-in, the score improves all 19 tested detectors on DF26 by 6.7% to 23.7% AUROC. The authors also introduce CaDet, which learns decoding traces from real-video/VAE-reconstruction pairs and combines them with a separate semantic path using DINOv3 ViT-L/16. CaDet reaches 95.8% AUROC on AIGVDBench’s closed-source generators; its generator-free version, trained without generated videos, reaches 90.9% on open-source and 91.9% on closed-source generators. The findings suggest temporal decoding traces can complement appearance-based detection, though strong resizing, frame-rate conversion, or re-encoding can weaken the signal.
Original abstract
Visible artifacts are an unreliable cue for detecting generated videos, because newer generators learn to remove them. We instead detect a trace left by the generation pipeline itself. Most video generators decode latents with a 3D causal VAE that groups consecutive frames in time, so the fidelity of generated frames rises and falls periodically with their position in a group. We call this periodic trace VAE Rhythm. Real videos are captured frame by frame and do not have it. To reveal it without training, we propose a phase scan. The scan reconstructs a clip with an off-the-shelf VAE from successive start offsets and measures the periodicity of the reconstruction error across these offsets. We find VAE Rhythm in most of the 39 variants of 31 generators we test, including the closed-source Sora, Kling 3.0 and Seedance 2.0. We use the rhythm as a zero-shot score, as a plug-in for existing detectors and as a data engine that pairs real videos with their VAE reconstructions to train our detector CaDet. On DF26, the training-free rhythm score outperforms all 15 retrained baselines, and as a plug-in it improves all 19 tested detectors by 6.7% to 23.7% AUROC. CaDet achieves the best AUROC on AIGVDBench, DF26 and three 2026 closed-source models, and its variant trained without any generated video still exceeds 90% on AIGVDBench.
Read the original paperMore in Computer Vision
Browse all 82 papers →InstantHMR : Real-Time Human Mesh Recovery on Your Phone
Mohamed Adjel, Kahina Chalabi, Vincent Bonnet
InstantHMR reconstructs detailed 3D human meshes from images in milliseconds, including on a smartphone.
Stochastic Deep Supervision for Gallery-Consistent Early-Exit Face Recognition
Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Naser Damer, Fadi Boutros
SD-ViT trains every ViT depth to produce useful face embeddings, enabling cheaper early-exit recognition without changing the gallery.
CoLa3D: Composable Latent 3D Decomposition
Xudong Cai, Yongcai Wang, Deying Li, Chuanxia Zheng
CoLa3D turns a whole-scene representation into a set of coherent, reusable 3D object meshes.