Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction

Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Prediction, developed by Shanghai AI Laboratory collaborators with universities including USTC, proposes FUTURE-L1, a Qwen3-VL-8B-based multimodal framework that alternates between text tokens and continuous latent visual spans instead of translating every future hypothesis into language. The core idea is to preserve dynamic future semantics in latent space for video event prediction, then train that behavior in two stages: supervised fine-tuning on FUTURE-L1-50K, a 50K subset curated from TwiFF-2.7M by selecting samples where intermediate future frames provide measurable visual gain, and LA-DAPO reinforcement learning, which adds outcome-contrastive and temporal-diversity rewards to optimize sampled latent trajectories. On FutureBench, FUTURE-L1 lifts Qwen3-VL-8B from 61.0 to 73.2 after SFT and to 85.4 after RL, surpassing Video-CoE by 10.4 points; on TwiFF-Bench, it raises the average score from 2.44 to 3.04. The ablations show the latent MSE weight of 0.1 and latent budget 4 are optimal in SFT, and the best RL coefficients are 0.2 for outcome-contrastive reward and 0.1 for temporal diversity. The paper also reports that FUTURE-L1-RL uses 195.3 tokens and 0.91 seconds per sample on FutureBench, indicating that its accuracy gains come with more compact inference than text-heavy multi-turn video reasoning.