ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing
AuthorsZhenghong Zhou, Zhe Lin, Jiebo Luo, Yuqian Zhou
AffiliationsUniversity of Rochester · Adobe Research*Advising authors.Project page
ALIVE makes inserted video objects behave as if they truly belong in the scene by learning how they should interact with existing actions.
Key results
Combined interaction-focused and general object-insertion video pairs.
Relative Overall-score improvement over the strongest evaluated baseline.
Relative Overall-score improvement over the strongest evaluated baseline.
Additional Overall-score points on the interaction benchmark from predicted temporal prompts.
What the paper found
ALIVE makes inserted objects respond to actions already unfolding in a video, rather than merely keeping them visible. The editor takes a source video, an edited first frame, and an object-only instruction; it uses an LTX-2.5 diffusion backbone, fine-tuned with LoRA, to infer the object’s subsequent behavior while preserving the surrounding scene. Its 35,800 training pairs combine rendered, real-world, and ROSE editing videos with model-generated examples made using Wan2.2; GPT helps design generation prompts, and Qwen3.6-27B checks pair quality. A Qwen3.6-27B vision-language model also predicts temporally localized interaction descriptions from the editing inputs, and ALIVE routes each description to the corresponding video interval through cross-attention. On the 128-case interaction benchmark, ALIVE improves the Overall score over the strongest baseline by 43.9%; on the general insertion benchmark, the gain is 4.4%. Predicted interaction guidance adds a further 0.95 points on the interaction benchmark without requiring users to describe the action. The results show that interaction-focused training and time-aligned guidance can make inserted objects behave more like part of the video’s scene.
Original abstract
Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated. We introduce ALIVE, a framework that makes inserted objects "alive" through coherent interactions with the source video's contents, using an edited first frame and an instruction naming only the added object. We curate 35,800 editing pairs combining 3D-rendered, model-generated, and real-world videos with general editing pairs from ROSE. Each pair differs in the target object's presence while preserving the surrounding action, teaching editors coordinated object behavior and source preservation. We further train a vision-language model (VLM) to predict interaction guidance from the same inputs. We introduce the ALIVE-interaction benchmark to assess interaction fidelity, source preservation, and visual coherence using a unified VLM-based protocol, and evaluate on the general video object insertion benchmark. Without VLM guidance, ALIVE improves Overall over the strongest evaluated baseline by 43.9% and 4.4% on the two benchmarks, respectively. VLM-predicted guidance further improves the ALIVE-interaction score by 0.95 points without additional user inputs.
Read the original paperMore in Multimodal AI
Browse all 64 papers →DEPICT: Scoring Text-to-Image Alignment by Answer Agreement
Vasco Ramos, Sandra Godinho Silva, Joao Magalhaes, Ricardo Rei, Pedro Henrique Martins
DEPICT improves text-to-image evaluation by checking whether an image and its caption lead to the same answers, especially for tricky cases like negation, missing objects, and incorrect attributes.
OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video
Hongyu Li, Manyuan Zhang, Kaituo Feng, Shu Chen, Dian Zheng, Hao Li, Hao Yu, Zhangquan Chen, Zoey Guo, Ray Zhang, Shaofei Huang, Tianrui Hui, Linjiang Huang, Si Liu
OneSearch-VL uses an evidence graph to help a single agent visually ground, retrieve, and synthesize reliable answers across images and videos.
Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation
Mohammed Irfan Kurpath, Jaseel Muhammad Kaithakkodan, Sahal Shaji Mullappilly, Ivan Laptev, Hisham Cholakkal
A compact embedding model unifies text, speech, audio, images, video, and documents in one search space without sacrificing the original text capabilities.