Selecting Diverse SFT Traces Improves Post-RL Generalization

This paper shows that verified reasoning traces are not interchangeable in the supervised fine-tuning, or SFT, stage before reinforcement learning with verifiable rewards. Instead of selecting similar solutions, it builds a rule-based topology fingerprint from reasoning-step types, transitions, branching, checking, and path statistics, then uses clustered farthest-point sampling to maximize route diversity at the same data budget. On RLVE, diverse SFT improved OLMo3-7B’s post-RL pass@8 coverage on environments held out from SFT by 16.9%, including problems harder than both training stages. The advantage was not dependent on multiple teachers: with Qwen3-4B-Thinking-2507 generating every candidate, diverse selection improved mean pass@8 across 10 mathematics benchmarks by up to 6.2%. The proposed mechanism is that route diversity places successful attempts within sampling reach on more prompts, producing mixed rewards that group-relative RL can learn from; before RL, mixed outcomes occurred on 54.7% of mathematics prompts versus 46.9% for similar SFT. The selector also worked on OpenThoughts3, INTELLECT-3, and Nemotron-Cascade 2, outperforming random, topology, gradient, embedding, and lexical baselines without model calls. On pools of about 2.1M solutions, it ran in about 3 hours on one CPU node, compared with 64–232 GPU-hours for more expensive alternatives. The results complement reasoning-model efforts from Qwen and DeepSeek by identifying route diversity, rather than SFT accuracy alone, as a practical way to prepare models for broader post-RL generalization.