NTH

Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation

AuthorsJaewon Cho, Minseo Yoon, Jinyoung Park, Jinwoo Seo, Jaewon Chu, Alice Oh, Hyunwoo J. Kim

AffiliationsKAIST · Korea University

November 14, 2026 2 min read
Watch on YouTube
The one-line take

The paper teaches multilingual reasoning models only on parts of their own reasoning where a teacher can still reliably guide them.

Key results

10
Languages evaluated

PolyMath High evaluation languages.

5,000
Training examples

OpenThoughts examples sampled across the evaluated languages.

11.60%
Qwen3-1.7B Avg@3

TOPSD score; COPSD scores 9.95%.

25.12%
Qwen3-8B Avg@3

TOPSD score, 2.64 percentage points above the strongest baseline.

35.28%
Qwen3-8B Pass@3

TOPSD score, 3.36 percentage points above the strongest baseline.

95%
Cutoff detection rate

TOPSD detects cutoffs in more than this share of trajectories across model scales.

What the paper found

This paper identifies Teacher Drift in on-policy self-distillation: as a student’s reasoning accumulates errors, its privileged teacher can begin echoing those errors instead of correcting them, and this drift tends to emerge earlier in lower-resource languages. The proposed Teachable On-Policy Self-Distillation, or TOPSD, uses the cumulative teacher–student likelihood ratio to select a teachable prefix, then checks whether the teacher can recover the correct answer from that prefix. Distillation is applied only to verified prefixes on the original student trajectory. On PolyMath High, the method was tested with Qwen3-1.7B, Qwen3-4B, and Qwen3-8B using 5,000 OpenThoughts examples across 10 languages. For Qwen3-1.7B, TOPSD reaches 11.60% Avg@3 versus 9.95% for COPSD; with Qwen3-8B, it achieves 25.12% Avg@3 and 35.28% Pass@3, gains of 2.64 and 3.36 percentage points over the strongest baselines. Cutoffs are detected in more than 95% of trajectories across model scales. The results show that effective multilingual distillation depends not only on teacher strength, but also on identifying when its guidance remains reliable.

Original abstract

On-Policy Self-Distillation (OPSD) improves reasoning by training a student on its own reasoning, using the same model conditioned on reference solutions as a teacher. Applying OPSD to multilingual reasoning requires the teacher to provide useful feedback across languages, even when the student makes mistakes in intermediate steps. We observe a phenomenon we call Teacher Drift: as these mistakes accumulate, the teacher may follow the student's incorrect steps instead of correcting them. Our analysis shows that this shift tends to occur earlier in lower-resource languages. To account for these differences across languages, we propose Teachable On-Policy Self-Distillation (TOPSD), which determines where and whether teacher supervision should be applied to each reasoning trajectory. TOPSD uses teacher-student likelihood to estimate where drift begins and select a prefix for supervision. It then verifies whether the teacher can recover the correct answer from that prefix, applying distillation only to prefixes that pass verification. Experiments on multilingual mathematical reasoning across ten languages and three model scales show that TOPSD consistently outperforms conventional OPSD, particularly in low- and medium-resource languages.

Read the original paper

More in Large Language Models

Browse all 99 papers →
03Llm

Cross-Primitive Consistency in Jev

Xu Liu, Guikun Chen, Kanzhi Wu, Wenguan Wang

PrimaCons measures when LLM decisions change across equivalent query formats, and CPRC helps make those decisions more consistent.

Read analysis