NTH

FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation

AuthorsYijing Lin, Zhendong Mao, Mengqi Huang, Yuxin Wang, Yongdong Zhang

AffiliationsIntroduction With recent advances in generative modeling [8, 19, 21, 33, 35, 57, 67], foundation models such as Stable Diffusion [13, 14, 41] and FLUX [25] offer strong text‑to‑image generation ca

November 10, 2026 2 min read
Watch on YouTube
The one-line take

FrameForge uses video models’ sense of temporal change to better generate images that preserve a visual input while following new instructions.

Key results

510K
Training examples

Total examples used across the three generation tasks.

182M
Trainable parameters

Parameters in the shared LoRA adaptation.

0.739
DreamBench overall

Overall score for subject-driven generation.

0.447
COCO2017 SSIM

Structural similarity score for canny-to-image generation.

6.12
GEdit-Bench overall

Overall score for instruction-guided image editing.

6.84
Motion-change score

GEdit-Bench score for motion-change editing.

What the paper found

FrameForge adapts the HunyuanVideo1.5 video-generation model for one shared image generator spanning subject-driven creation, canny-guided synthesis, and instruction-based editing. Its Mixture of Temporal Experts selects among virtual temporal-position embeddings according to each input and denoising step, while Chain of Temporal Reasoning adds a latent intermediate pathway that encourages the model to represent how a condition becomes a target. The backbone stays frozen, and a shared LoRA is trained on 510K examples. On DreamBench, FrameForge scores 0.739 overall; on COCO2017, its SSIM is 0.447, a 7.2% improvement over the next-best method. On GEdit-Bench, it reaches 6.12 overall and 6.84 for motion changes, exceeding the open-source comparison while its motion-change score remains below GPT-4o but above Gemini 2.5 and Seedream 4.0. The model has 182M trainable parameters, and ablations show that combining adaptive temporal selection with the reasoning pathway improves results across the three tasks.

Original abstract

Unified conditional image generation requires a single model to preserve different aspects of a visual input while producing a target image that follows textual guidance. Pretrained video models provide a promising foundation by encoding visual transformations across time, yet existing approaches typically assign the condition and target to fixed temporal positions. We observe that condition-target relationships vary substantially across generation tasks, while video models naturally encode visual transformations at different temporal scales. This motivates us to model conditional generation through input-adaptive virtual temporal relations. We propose FrameForge, which captures two complementary aspects of such relations. Mixture of Temporal Experts (MoTE) models how far the target lies from the condition by dynamically selecting temporal RoPE experts with different virtual intervals. Chain of Temporal Reasoning (CoTR) models how the condition is transformed into the target through a latent intermediate state, with a reasoning-bottleneck attention mask encouraging information propagation through this pathway. Experiments on DreamBench, COCO2017, and GEdit-Bench show that FrameForge achieves superior performance in subject-driven generation, canny-to-image generation, and image editing among the compared methods.

Read the original paper

More in Generative Models

Browse all 87 papers →
01Generative Model

dNaViT: Discrete Codes are General-Purpose Visual Representations

Xiaotong Li, Zhengrong Yue, Xu Huang, Jialv Zou, Minhao Jing, Wenhao Yan, Haodong Li, Hai Huang, Bo Zhao, Yan Feng, Peng Pei, Xunliang Cai

dNaViT turns images into reusable discrete codes designed to work across visual understanding, generation, and editing.

Read analysis