Selection Is Not Learning: Choosing Model Futures
AuthorsHaoyi Yang, Kshitij Mishra, Nils Lukas, Salem Lahlou
AffiliationsMohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
A framework for self-evolving language models that checks whether an update actually helps before inheriting it, while retaining useful experience from other updates.
Key results
Share of audited updates on verified tasks that did not improve correct-target negative log-likelihood.
HEIR COMMIT reduction relative to random inheritance at the same acceptance rate.
EVO HEIR’s reported relative math-average improvement across four backbones.
EVO HEIR beats Agent0 on 45 of 48 comparisons.
Average improvement in percentage points using the same references.
What the paper found
Self-evolving systems such as R-Zero, Agent0, and Skill Self-Play often select tasks that appear learnable, then inherit every resulting update. The paper finds that task selection does not reliably predict whether training helps: 62.5% of audited Skill Self-Play updates on verified tasks did not improve correct-target negative log-likelihood. HEIR COMMIT addresses this selection-to-update gap by freezing a proposed successor and committing it only if it beats the current model on fresh evaluation probes. Across three hosts, it reduces the harmful share of accepted updates by 68–83%. EVO HEIR goes further by separating parameter inheritance from experience inheritance: it screens for a model recipient, uses current and archived solutions to repair that recipient’s failures while preserving successes, then independently confirms the resulting model. Across Qwen3, OLMo-3, and Google’s Gemma-4-E4B-it, EVO HEIR raises the math average by 5.3% relative to Agent0 and beats Agent0 on 45 of 48 benchmark–backbone comparisons. Recovery also outperforms direct fine-tuning on the same references by 1.93 percentage points on average, showing that useful experience can persist even when its source model is rejected.
Original abstract
Self-evolving language models train on tasks they generate, so every round must decide which updated model the next round inherits. Current loops such as R-Zero, Agent0, and Skill Self-Play make this decision through the task, admitting tasks that look learnable and inheriting every resulting update. We identify a selection-to-update gap in these loops, where a task that looks learnable need not produce a useful update. Most updates on verified tasks fail to make correct answers more likely, yet these loops inherit them all. HeirCommit freezes the updated model and uses fresh evaluation questions to decide whether to inherit it or retain the current model. EvoHeir separates parameter inheritance from experience inheritance, using current and archived experience to repair failures and preserve successes before final confirmation. Our analysis characterizes selection-induced optimism and decomposes regret into selection, absorption, and confirmation errors. HeirCommit cuts the harmful share of accepted updates by 68-83% relative to random inheritance at the same acceptance rate. Across four backbones, EvoHeir achieves the best mathematics and general-reasoning averages, raising the math average by 5.3% relative to the strongest baseline, Agent0. The framework lets useful experience persist even when the model that produced it is not inherited.
Read the original paperMore in Continual Learning
Browse all 27 papers →Cortex: Consolidating Context into Parametric Memory
Jintian Zhang, Feng He, Xinnian Liang, Ziwen Xu, Haoming Xu, Haojing Huang, Shuangzhi Wu, Ningyu Zhang
Cortex aims to help language models retain useful information from temporary context without forgetting what they already know.
MemT-1.0: Towards Memory-Native Evolution
Zhiyu Li, Zeyu Zhang, Linyun Liu, Yiwei Bu, Chenyang Xi, Xichong Zhang, Yi Yu, Yu Yu, Zhichen Liu, Yuxiao Yi, Chunyu Li, Haoran Liu, Zhenkai Ding, Le Zhou, Juncheng Zhang, Boyu Chen, Zehao Lin, Guorui Li, Renyu Fu, Ji Li, Youyang Xia, Jiajun Shen, Harvey Li, Jingsong Zhou, Helan Hu, Tong Shen, Tianxiao Xu, Chenye Han, Weijie Li, Zhongquan Zhou, Hongming Xu, Yuekeng Li, Erenkai Zhu, Peng Yu, Wenhao Li, Jingyi Kang, Xixuan Hao, Xuzhou Zhu, Can Chen, Jiawei Liu, Chen Jiang, Tao Jiang, Fangwei Xiang, Zepeng Yao, Kai Chen, Guang Yang, Mingfei Lu, Chunhui Li, Zhengxuan Jiang, Bo Tang, Shiyu Zhan, Yihang Sun, Jie Chu, Wenjun Xiong, Zhenxiang Hua, Chen Lin, Detong Su, Dongyu Deng, Qiuping Ding, Guofeng Ding, Si Chen, Guohua Wen, Xu Chen, Zhi-Qin John Xu, Ying-Cong Chen, Ningyu Zhang, Haofen Wang, Muning Wen, Haotian Chen, Wentao Zhang, Xuanhe Zhou, Rita Zhang, Junchi Yan, Feiyu Xiong
MemT-1.0 builds persistent memory into language models to help them retain and use information more efficiently over long histories.
ASCENT: Online Test-Time Training of Long-Horizon Agents via Self-Distillation of Verified Experience
Haodong Lu, Dong Gong
ASCENT lets deployed LLM agents learn from verified successes on the fly by converting hindsight about their own trajectories into lasting weight updates.