NTH

Route-DMD: Distribution Matching Distillation with Task-Routed Mechanism for Speech Foundation Models

AuthorsYanru Huo, Zhikang Niu, Wenming Tu, Junxi Liu, Tian Tan, Shan Yang, Yipeng Kang, Ziyang Ma, Xie Chen, Zilong Zheng

AffiliationsMoE Key Lab of Artificial Intelligence, X-LANCE Lab, Shanghai Jiao Tong University · State Key Laboratory of General Artificial Intelligence, BIGAI · Shanghai Innovation Institute · Tencent Hunyuan · Nov 2026 · Nanyang Technological University

November 15, 2026 2 min read
Watch on YouTube
The one-line take

Route-DMD aims to make a versatile speech model run much faster by tailoring its distillation objective to the task.

Key results

4
Student inference steps

The distilled AuK student uses 4 steps, compared with 32 for the teacher.

4.5
End-to-end speedup

The 4-step student achieves a 4.5× speedup over the 32-step teacher.

0.0958
Distilled real-time factor

Measured on an NVIDIA H20; the teacher’s real-time factor is 0.4333.

13.85
MMAE speech editing EMR

Exact match rate for the Route-DMD student.

What the paper found

Route-DMD tackles a problem in distilling speech foundation models: standard distribution matching can damage speech separation because noisy, incorrect student outputs lead the teacher outside the distribution it was trained on. The method first uses consistency distillation to initialize a fast student, then routes speaker and vocal separation to regression against ground-truth targets while retaining Decoupled DMD for generation, editing, and other processing tasks. It also uses adaptive projected guidance, or APG, instead of classifier-free guidance to reduce artifacts inherited from teacher rollouts. Tested on AuK, the student runs in 4 steps rather than the teacher’s 32 and achieves a 4.5× end-to-end speedup; its real-time factor falls from 0.4333 to 0.0958 on an NVIDIA H20. It remains competitive with the teacher on generation and editing, reaches an exact match rate of 13.85 on MMAE speech editing, and improves overall quality and UTMOS across CHiME-4, Libri2Mix, and VCTK-SR. The paper also compares its results with systems including Qwen3-TTS.

Original abstract

Speech foundation models built on flow matching handle generation, editing, and processing in one system, but every request costs 20-50 iterative steps, and classifier-free guidance (CFG) doubles each of them. Distribution matching distillation (DMD) trains a student that approaches the teacher's quality in a few steps, yet existing speech distillation work applies it to generation alone. We show that a distribution-matching objective does not serve every task: in separation, the noised states of erroneous student outputs deviate far from the teacher's training distribution, where its prediction is unreliable. Route-DMD therefore makes the objective task-dependent, routing the affected tasks to a regression loss and keeping decoupled distribution matching elsewhere. The student is initialized by consistency distillation, and teacher rollouts use adaptive projected guidance (APG) instead of CFG. Without CFG, the 4-step student matches the 32-step teacher on generation, editing, and processing tasks at a 4.5$\times$ end-to-end speedup.

Read the original paper

More in Speech AI

Browse all 48 papers →
02Speech

Marimba: A System One Model for Speech

Solano Amadori

Marimba reuses a speech recognition model’s features to answer questions about words, speakers, language, and turn endings.

Read analysis
03Speech

VoxMel TTS Technical Report

Wenhao Guan, Jiaxin Ye, Haitao Li, Yifan Duan, Junxi Liu, Wangjin Zhou, Lin Li, Qingyang Hong, Xie Chen

VoxMel generates speech by autoregressively modeling transformed mel-spectrograms, avoiding a learned tokenizer while aiming for high fidelity and faster inference.

Read analysis