Latent-MOPD: Latent Multi-Teacher On-Policy Distillation

Latent-MOPD introduces representation-level multi-teacher on-policy distillation for language models, extending MOPD beyond token probabilities by matching the hidden states that produce those predictions. A domain router sends each student-generated response to a math, code, or logic specialist, and the same teacher supplies both token-level policy-gradient targets and late-layer representation targets. For same-family models, the method aligns the final three layers directly; for cross-family teachers, it uses a shared trainable projection to bridge hidden widths. Domain-pure optimizer updates prevent conflicting representation gradients, while a per-teacher crossfade gradually replaces latent supervision with token supervision. Experiments distill three specialists into the 1.5B DeepSeek-R1-Distill-Qwen student, using 17,856 prompts from DAPO-Math-17k, OpenCodeReasoning, and Reasoning Gym. Across nine math, coding, and logic benchmarks, Latent-MOPD beats token-only, representation-only, and uniform-averaging baselines, reaches a normalized score of 1.05, and surpasses the strongest individual teacher on five benchmarks despite matching each teacher’s parameter count. With 7B Qwen-based teachers, including NVIDIA’s AceReason-Nemotron and DeepSeek’s R1-Distill-Qwen, the method also improves on all six benchmarks; AIME24 rises to 39.4 from 36.5 for token-only MOPD. Training lasts 62 optimizer steps, and the merged-initialization experiment improves five of six benchmarks, suggesting that latent supervision can consolidate specialist capabilities into one deployable student without retaining teachers or projection layers at inference.