Route-DMD: Distribution Matching Distillation with Task-Routed Mechanism for Speech Foundation Models
AuthorsYanru Huo, Zhikang Niu, Wenming Tu, Junxi Liu, Tian Tan, Shan Yang, Yipeng Kang, Ziyang Ma, Xie Chen, Zilong Zheng
AffiliationsMoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · State Key Laboratory of General Artificial Intelligence, BIGAI · Shanghai Innovation Institute · Tencent Hunyuan · Nov 2026 · Nanyang Technological University
Route-DMD aims to make a versatile speech model run much faster by tailoring its distillation objective to the task.
Key results
The distilled AuK student uses 4 steps, compared with 32 for the teacher.
The 4-step student achieves a 4.5× speedup over the 32-step teacher.
Measured on an NVIDIA H20; the teacher’s real-time factor is 0.4333.
Exact match rate for the Route-DMD student.
What the paper found
Route-DMD tackles a problem in distilling speech foundation models: standard distribution matching can damage speech separation because noisy, incorrect student outputs lead the teacher outside the distribution it was trained on. The method first uses consistency distillation to initialize a fast student, then routes speaker and vocal separation to regression against ground-truth targets while retaining Decoupled DMD for generation, editing, and other processing tasks. It also uses adaptive projected guidance, or APG, instead of classifier-free guidance to reduce artifacts inherited from teacher rollouts. Tested on AuK, the student runs in 4 steps rather than the teacher’s 32 and achieves a 4.5× end-to-end speedup; its real-time factor falls from 0.4333 to 0.0958 on an NVIDIA H20. It remains competitive with the teacher on generation and editing, reaches an exact match rate of 13.85 on MMAE speech editing, and improves overall quality and UTMOS across CHiME-4, Libri2Mix, and VCTK-SR. The paper also compares its results with systems including Qwen3-TTS.
Original abstract
Speech foundation models built on flow matching handle generation, editing, and processing in one system, but every request costs 20-50 iterative steps, and classifier-free guidance (CFG) doubles each of them. Distribution matching distillation (DMD) trains a student that approaches the teacher's quality in a few steps, yet existing speech distillation work applies it to generation alone. We show that a distribution-matching objective does not serve every task: in separation, the noised states of erroneous student outputs deviate far from the teacher's training distribution, where its prediction is unreliable. Route-DMD therefore makes the objective task-dependent, routing the affected tasks to a regression loss and keeping decoupled distribution matching elsewhere. The student is initialized by consistency distillation, and teacher rollouts use adaptive projected guidance (APG) instead of CFG. Without CFG, the 4-step student matches the 32-step teacher on generation, editing, and processing tasks at a 4.5$\times$ end-to-end speedup.
Read the original paperMore in Speech AI
Browse all 48 papers →Kezek: Splitting Mono Two-Person Conversations into Per-Speaker Tracks for Full-Duplex Dialogue Data
Bagzhan Karl, Alizhan Karl
Kezek turns a mixed two-person recording into speaker-specific tracks while preserving overlaps and short replies for full-duplex dialogue data.
Marimba: A System One Model for Speech
Solano Amadori
Marimba reuses a speech recognition model’s features to answer questions about words, speakers, language, and turn endings.
VoxMel TTS Technical Report
Wenhao Guan, Jiaxin Ye, Haitao Li, Yifan Duan, Junxi Liu, Wangjin Zhou, Lin Li, Qingyang Hong, Xie Chen
VoxMel generates speech by autoregressively modeling transformed mel-spectrograms, avoiding a learned tokenizer while aiming for high fidelity and faster inference.