Sample-Efficient Learning from Agent Experience

Researchers from Monash University and ByteDance Seed introduce Experience Distillation, a method for consolidating an agent’s trial-and-error learning into model weights without collecting additional environment data. The approach uses one-step branched rollouts: at histories already present in recorded trajectories, an experience-conditioned teacher generates only its next action, and a student learns that action through sampled-token next-token prediction. Experience preprocessing, enhanced teacher reasoning, and branch packing make the procedure practical without a learned world model, avoiding compounding synthetic-rollout errors. Across 749 curated software-engineering tasks and six TaleSuite text-adventure games, Experience Distillation reaches 51.4% pass@1 on software engineering and retains 64.8% of the improvement that in-context learning provides, while direct supervised fine-tuning retains only 3.8%. On TaleSuite, it achieves an average normalized score of 43.8 and matches classical reinforcement-learning performance with 9.6× fewer environment samples on software engineering and 57.2× fewer on the games. The method also generalizes to 494 out-of-distribution software tasks and supports continual collect-and-distill cycles, suggesting a workflow in which agents first adapt rapidly in context and then permanently internalize useful experience.