Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers

Cross-Space Distillation, from Qualcomm AI Research with collaborators at the University of Wisconsin–Madison, Johns Hopkins University, and Rutgers University, reframes one-step diffusion distillation for the real deployment case where teacher and student do not share a latent space. The paper shows that modern teachers such as Stable Diffusion 3.5, FLUX.2-klein-4B, SDXL, Kolors, and PixArt-Σ can be distilled into compact Stable Diffusion 1.5 and SD 2.1 students despite a 1024×1024 teacher-to-512×512 student resolution gap and mismatched VAE parameterizations. The key module, Bridge, is a lightweight 5M-parameter latent interface that reuses a frozen prefix of the student decoder as a spatial prior and learns a SwinIR projection head, trained with L1 latent reconstruction and an attention fidelity loss using reverse-KL on teacher self-attention. Across five teachers, the SD 1.5 student improves HPSv3 from 5.37 to 9.42 with SD 3.5 Medium, while the merged checkpoint reaches 10.53 HPSv3 and 29.07 HPSv2 without increasing the 1-step inference budget. The supplementary report shows Bridge adds only 6.01 ms to 18.01 ms runtime and 0.54 GB to 0.78 GB peak memory, and that a 30% pruned 1.5B baseline is far weaker than the 0.86B Bridge student on HPSv3, HPSv2, ImageReward, MPS, and DPG Bench. The method also supports inference-time resolution upgrade, decoding 512×512 student outputs into teacher-compatible 1024×1024 images.