Knowing When Thinking Is Not Enough: Teaching Small Reasoning Models to Reason Beyond Their Parametric Knowledge

This paper argues that small reasoning models should not automatically think longer: their failures split into execution bottlenecks, where self-refinement can recover an already reachable solution, and knowledge bottlenecks, where missing external information is required. Interventions across Qwen3 and Gemma models show that epistemic verbalizations such as “wait” mostly concentrate probability on existing solution paths rather than creating new ones. The proposed FlyBy framework trains Qwen3-based 4B and 8B models to reason first, diagnose the bottleneck, and selectively query stronger backends without revealing the original problem. Supervised fine-tuning teaches query and answer-integration actions, while cost-aware reinforcement learning learns when to query, what to ask, and whether to use shallow or deep assistance from models including DeepSeek-V4-Pro. On 1,158 hard problems spanning ArXivMath, GPQA-Diamond, SuperGPQA, ChemBench, MedXpertQA, and MMLU-Pro, FlyBy-4B reaches 45.96% pass@8, surpassing Qwen3-14B at 2.7 times lower serving cost; it also reaches 16.85% pass@1, above Qwen3-8B’s 15.31%. Scaling to FlyBy-8B raises pass@8 to 51.81%. The learned policy performs targeted, sometimes repeated queries rather than outsourcing full solutions, and transfers to alternative backends such as OpenAI’s GPT-5.6 Luna, indicating that the key capability is diagnosing when internal reasoning is insufficient.