Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation
AuthorsJaewon Cho, Minseo Yoon, Jinyoung Park, Jinwoo Seo, Jaewon Chu, Alice Oh, Hyunwoo J. Kim
AffiliationsKAIST · Korea University
The paper teaches multilingual reasoning models only on parts of their own reasoning where a teacher can still reliably guide them.
Key results
PolyMath High evaluation languages.
OpenThoughts examples sampled across the evaluated languages.
TOPSD score; COPSD scores 9.95%.
TOPSD score, 2.64 percentage points above the strongest baseline.
TOPSD score, 3.36 percentage points above the strongest baseline.
TOPSD detects cutoffs in more than this share of trajectories across model scales.
What the paper found
This paper identifies Teacher Drift in on-policy self-distillation: as a student’s reasoning accumulates errors, its privileged teacher can begin echoing those errors instead of correcting them, and this drift tends to emerge earlier in lower-resource languages. The proposed Teachable On-Policy Self-Distillation, or TOPSD, uses the cumulative teacher–student likelihood ratio to select a teachable prefix, then checks whether the teacher can recover the correct answer from that prefix. Distillation is applied only to verified prefixes on the original student trajectory. On PolyMath High, the method was tested with Qwen3-1.7B, Qwen3-4B, and Qwen3-8B using 5,000 OpenThoughts examples across 10 languages. For Qwen3-1.7B, TOPSD reaches 11.60% Avg@3 versus 9.95% for COPSD; with Qwen3-8B, it achieves 25.12% Avg@3 and 35.28% Pass@3, gains of 2.64 and 3.36 percentage points over the strongest baselines. Cutoffs are detected in more than 95% of trajectories across model scales. The results show that effective multilingual distillation depends not only on teacher strength, but also on identifying when its guidance remains reliable.
Original abstract
On-Policy Self-Distillation (OPSD) improves reasoning by training a student on its own reasoning, using the same model conditioned on reference solutions as a teacher. Applying OPSD to multilingual reasoning requires the teacher to provide useful feedback across languages, even when the student makes mistakes in intermediate steps. We observe a phenomenon we call Teacher Drift: as these mistakes accumulate, the teacher may follow the student's incorrect steps instead of correcting them. Our analysis shows that this shift tends to occur earlier in lower-resource languages. To account for these differences across languages, we propose Teachable On-Policy Self-Distillation (TOPSD), which determines where and whether teacher supervision should be applied to each reasoning trajectory. TOPSD uses teacher-student likelihood to estimate where drift begins and select a prefix for supervision. It then verifies whether the teacher can recover the correct answer from that prefix, applying distillation only to prefixes that pass verification. Experiments on multilingual mathematical reasoning across ten languages and three model scales show that TOPSD consistently outperforms conventional OPSD, particularly in low- and medium-resource languages.
Read the original paperMore in Large Language Models
Browse all 99 papers →All Quiet on the Neural Front: Language Models Do Not Report Injected Pain States
Emilio Ferrara
Models’ internal signals can reveal an injected pain-like state and change their behavior, even when the models themselves do not report feeling it.
On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation
Yichen Wang, Yufa Zhou, Siqi Zhu, Haozheng Luo, Jingzhi Bao, Zhichao Liu, Zijun Wei
Distilling a model’s skills can preserve benchmark performance while losing its ability to generalize, and this paper explores ways to prevent that.
Cross-Primitive Consistency in Jev
Xu Liu, Guikun Chen, Kanzhi Wu, Wenguan Wang
PrimaCons measures when LLM decisions change across equivalent query formats, and CPRC helps make those decisions more consistent.