AutoEnvScaling: Automating the Data Flywheel with Terminal Agents
AuthorsSimon Yu, Baolin Peng, Bo Liu, Zhengyan Shi, Zhaoyang Wang, Hao Cheng, Qianhui Wu, Wenlin Yao, Natasha Jaques, Weiyan Shi, Jianfeng Gao
AffiliationsNortheastern University Microsoft Stanford University · University of Washington University of North Carolina at Chapel Hill Project Page Code
AutoEnvScaling has agents create the training environments that help terminal agents get better at solving tasks.
Key results
Up to a 25-fold increase over prompt-only generation.
Maximum reduction compared with prompt-only generation.
Points above the matched Tmax baseline with GPT-5.6-sol as proposer.
Points gained when the model serves as both proposer and solver.
Final score, increased from 1.6.
What the paper found
AutoEnvScaling turns environment design into a terminal-agent task: a proposer uses a sandbox, coding tools, and the solver’s past rollouts to build complete Harbor environments, then runs and revises them before admission. Accepted tasks are validated against reference and empty solutions and calibrated to challenge the current solver; its later successes and failures guide the next generation, forming an adaptive data flywheel. Across eight frontier models, this approach produces up to 25-fold more valid environments than prompt-only generation while cutting cost per valid environment by up to 75%. With GPT-5.6-sol as proposer, training Qwen3.5-35B-A3B on these evolving environments beats a matched static Tmax pool by 4.3 points on Terminal-Bench 2.1. The stronger self-improvement result uses Qwen3.6-35B-A3B as both proposer and solver: its Terminal-Bench 2.1 score rises by 12.2 points, and its Terminal-Bench 4.0 score climbs from 1.6 to 3.2. The proposer and solver can be jointly trained with Divergence Proximal Policy Optimization, while the proposer also improves its persistent harness. GPT-5.6-sol and Claude Opus 5 serve as external proposers in experiments, and generated environments span software engineering, computer use, professional workflows, and GPU kernels. The results show that agents can help build their own reinforcement-learning curricula, though the study does not establish that recursive gains continue with longer training or that cross-domain generation improves downstream solving.
Original abstract
Terminal agents solve tasks in software engineering and scientific research by executing commands in a terminal. Training them via reinforcement learning (RL) requires diverse and verifiable environments, but these environments are still mostly designed by human researchers, so their supply grows only as fast as researchers can work. We introduce AutoEnvScaling, which treats environment design as a terminal task and uses agents to automate the data flywheel. A proposer agent builds new RL environments from the rollouts of the solver being trained, and the solver's new rollouts guide the next round. By turning environment generation into a terminal task, AutoEnvScaling increases the valid-environment rate by up to 25 times compared to prompt-only generation, at up to 75% lower cost per valid environment. We show that training Qwen3.5-35B-A3B on AutoEnvScaling environments from GPT-5.6-sol improves its Terminal-Bench 2.1 score by 4.3 points over the matched Tmax baseline. The broader goal of AutoEnvScaling is to enable recursive self-improvement (RSI): by making both environment generation and solving terminal tasks, it lets one model co-evolve in both roles. We show that Qwen3.6-35B-A3B, acting in both roles, gains 12.2 points on Terminal-Bench 2.1 and raises its Terminal-Bench 4.0 score from 1.6 to 3.2. The pipeline also generalizes across agentic domains, generating environments for software engineering, computer use, professional work, and GPU kernel programming. Overall, these results show a path toward agents building their own training environments, reducing human reliance and taking a core step toward RSI.
Read the original paperMore in AI Agents
Browse all 65 papers →DuoDiscover: Scaling Autonomous Discovery Beyond the Evaluation Bottleneck
Shubham Agarwal, Akshat Gupta, Mike Heddes, Dilara Soylu, Rui Meng, Patrick Tser Jern Kon, Jiefeng Chen, Mert Cemri, Shu Liu, Abe Gonzalez, Akanksha Jain, Bhavana Dalvi Mishra, Linfeng Song, Vilobh Meshram, Xiangru Tang, Jinsung Yoon, Aditya Parameswaran, Ion Stoica, Vahab Mirrokni, Burak Gokturk, Parthasarathy Ranganathan, Tomas Pfister, Nanyun Peng, Chun-Liang Li
DuoDiscover uses a fleet of AI agents and fast, continuously checked proxy tests to find engineering improvements when full evaluations are too slow.
World of SWECraft: Agent Collaboration at Scale via Simulated Software Worlds
Shannon Zejiang Shen, Yapei Chang, Qiyao Liang, Benjamin Newman, Jenny Liang, Rulin Shao, Ziqiao Ma, Junjie Oscar Yin, Yung-Sung Chuang, Weijia Shi, Yejin Choi, Luke Zettlemoyer, Pang Wei Koh, Mohit Iyyer, Tongshuang Wu, David Sontag
The paper builds simulated software ecosystems to test how AI agents collaborate and scale their work.
CUAWright: A Minimal Unified Interface for Digital Agents
Yadong Lu, Theodore Lee, Yifei Li, Lawrence Keunho Jang, Tianci Xue, Yu Su, Huan Sun, Ahmed Hassan Awadallah
CUAWright argues that giving computer-use agents a simple programmable terminal instead of GUI-specific tools makes them substantially more capable, efficient, and adaptable.