NTH

StepX-OPD: Rethinking Multi-Teacher On-Policy Distillation in Diffusion Models

AuthorsChenru Wang, Xuanyang Zhang, Yucheng Han, Wenbo Nie, Wei Cheng, Xianfang Zeng, Gang YU, Chi Zhang

AffiliationsStepFun

November 10, 2026 2 min read
Watch on YouTube
The one-line take

StepX-OPD trains diffusion models to combine specialist capabilities and generate images in just a few steps, without the usual separate compression stage.

Key results

4
Sampling steps

StepX-OPD inference setting evaluated on SD3.5-Medium and FLUX.1-dev.

0.8901
SD3.5-Medium normalized average

StepX-OPD score across GenEval, OCR, DeQA, and PickScore.

0.8902
FLUX.1-dev normalized average

StepX-OPD score across GenEval, OCR, DeQA, and PickScore.

37%
GPU-hour reduction

Reduction versus sequential DiffusionOPD followed by TDM on SD3.5-Medium.

94
SD3.5-Medium prompt-word recurrence

Images with prompt-word recurrence for StepX-OPD, compared with 218 for DiffusionOPD.

84
Precise-teacher prompt-word recurrence

Recurrence count when StepX-OPD uses the optional precise teacher target.

What the paper found

StepX-OPD combines multi-teacher capability integration and few-step distillation in one training run, rather than first building a full-step teacher and compressing it later. On SD3.5-Medium and FLUX.1-dev, frozen specialists supervise states from the student’s own randomized rollout; source routing matches each training batch to the relevant specialist, while interpolated rollout states and exact re-noising provide supervision between steps. The method pairs trajectory distribution matching with velocity regression concentrated at low noise, where the paper’s reverse-KL analysis assigns greater weight to velocity mismatch. At 4 sampling steps, StepX-OPD achieves normalized average scores of 0.8901 on SD3.5-Medium and 0.8902 on FLUX.1-dev, the highest among the compared four-step methods and higher than the forty-step Flow-OPD model. On SD3.5-Medium, it uses about 37% fewer GPU-hours than sequential DiffusionOPD followed by TDM; training runs use NVIDIA H800 GPUs. The authors also measure unintended prompt-word rendering: on SD3.5-Medium, recurrence rises from 21 images for the base model to 218 for DiffusionOPD, while StepX-OPD reduces it to 94, or 84 with an optional more precise teacher target.

Original abstract

A deployable text-to-image diffusion model must hold several capabilities at once, such as instruction following, text rendering and human preference, and must sample in a few steps. Capabilities and few-step sampling are built in separate stages: a full-step model first consolidates capabilities through multi-teacher on-policy distillation, then serves as a fixed teacher for step distillation. Every full-step model we evaluate scores lower once compressed, and the second stage cannot begin until the first finishes. We introduce StepX-OPD, which merges the two stages: one student acquires every capability while learning to sample in a few steps, so frozen specialists supervise it on the states its own rollout visits rather than on a longer trajectory it never runs. Because a decomposition of the rollout's KL weights velocity mismatch most heavily near the clean end, we also regress the student's velocity on the low-noise part of the rollout. Separately, we find that capability integration makes models write prompt words into images on prompts that never asked for text, and we give a variant that lowers it. We evaluate on SD3.5-Medium and FLUX.1-dev across instruction following, text rendering, image quality and human preference. On both architectures, StepX-OPD attains the highest normalized average score among the four-step methods compared and exceeds the forty-step Flow-OPD model, and on SD3.5-Medium it trains with about 37% fewer GPU-hours than sequential DiffusionOPD followed by step distillation.

Read the original paper

More in Diffusion Models

Browse all 82 papers →