From Repositories to Training Environments: Scaling Long-Horizon Coding Agents with Environment Evolution Loop
AuthorsYang Tian, Tianyu Zheng, Xiyuan Yang, Yaxin Du, Chengran Yang, Bo Zhao, Jian Yang, Bryan Dai
AffiliationsBeihang University School of AI, Shanghai Jiao Tong University IQuest Research ∗ † Project Leader
RepoFactory turns real software repositories into training environments and iteratively improves them to help coding agents tackle longer, more realistic projects.
Key results
Verifiable full-repository reconstruction environments produced.
Domains covered by the generated environments.
Percentage-point gain over base Qwen3.5-35B-A3B.
Percentage-point gain across five benchmarks over the base model.
Additional percentage points versus training without environment evolution.
Additional average percentage points versus training without environment evolution.
What the paper found
RepoFactory turns real Python repositories into full-repository reconstruction environments, where an agent must build a working project from a language specification. Its Environment Evolution Loop iteratively revises the specification, oracle implementation, and tests using a multiplicative objective over validity, structural coupling, and implementation scope—without requiring costly solver rollouts. The pipeline produced 10159 environments across 17 software domains; training Qwen3.5-35B-A3B on the resulting data created RepoHorizen-35B-A3B. On NL2Repo-Bench, the model scored 46.4, a gain of 22.6 percentage points over its base model, and improved by an average of 8.5 points across five coding and agentic benchmarks. Those scores exceed GPT-5.5 from OpenAI and Gemini-3.1 Pro on NL2Repo. Compared with training without environment evolution, the loop adds 5.1 points on NL2Repo and 2.1 points on average across the five benchmarks. The results show that repository environments can support broad, long-horizon coding-agent training, while low-cost structural feedback can improve both the number of usable tasks and their training value.
Original abstract
Training coding agents for long-horizon tasks requires environments with verifiable feedback. Existing pipelines often follow a linear process that ends with validation, establishing usability but providing little guidance on improving training value. Recursive self-improvement (RSI) often requires model rollouts or model training followed by downstream evaluation, with costs and feedback delays amplified on long-horizon tasks. How can we use timely, reliable proxy signals to improve training value and synthesize environments at scale? We present RepoFactory, an automated pipeline that converts real repositories into verifiable environments where agents build complete projects from scratch given a language specification. Its Environment Evolution Loop optimizes a multiplicative objective combining validity, structural coupling, and implementation scope through proxy feedback. RepoFactory produces 10,159 environments across 17 software domains. Training Qwen3.5-35B-A3B on these environments yields RepoHorizen-35B-A3B, which achieves gains of 22.6 percentage points on NL2Repo-Bench and 8.5 points on average across five benchmarks covering long-horizon coding and agentic tasks in real-world production settings, compared with 17.5 and 6.4 points without environment evolution. These results highlight the training value of RepoFactory environments and the additional gains from the Environment Evolution Loop.
Read the original paperMore in Code Generation
Browse all 52 papers →CompoundingKernel: Self-Improving from Zero to First
Qian Qu, Hongyang Wang, Wenbo Chen, Yichuan Li, Jingguo Liu, Xian Li, Xuan Wang, Ruochen Wen, Leo Fan, Yiming Yan, Zheshen Wang, Benoit Dumoulin
CompoundingKernel uses parallel agents and an archive of successful attempts to discover fast, correct GPU kernels.
CoReMem: Cost-Internalizing Memory for Successive Coding Tasks with Prospective Valuation
Peiwen Li, Harshit Verma, Sophia Zhou, Haiwen Wang, Ngoc Bui, Heng Chang, Fei Fei, David van Dijk, Rex Ying
CoReMem helps coding agents carry forward useful repository knowledge while accounting for the cost of writing and using memory.
Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization
Gabriele Oliaro, Jaeseong Lee, Zhaoyuan Su, Sreeram Vennam, Zhihao Jia, Samyam Rajbhandari
KernelSeek shows how asking an AI coding agent the right question can help it build faster GPU kernels without handing it the answer.