Research on models that learn new tasks and information over time. Compare approaches to retaining knowledge, adapting, and reducing forgetting.
24 papers · Latest edition October 6, 2026
Three of the latest briefs in this collection. Read the evidence and the original papers alongside them.
ASCENT lets deployed LLM agents learn from verified successes on the fly by converting hindsight about their own trajectories into lasting weight updates.
SourceLearn helps LLM agents progressively build reusable expertise about trusted information sources instead of repeatedly starting from scratch.
Local Support Learning helps large language models learn new skills without overwriting what they already know by activating updates only where they are locally needed.
Newest editions first.
Haodong Lu, Dong Gong
ASCENT lets deployed LLM agents learn from verified successes on the fly by converting hindsight about their own trajectories into lasting weight updates.
Lucheng Fu, Kejing Xia, Yiyang Wang, Yiqiao Jin, Jinjin He, Xiyuan Yang, Haoxin Liu, Ye Yu, Haibo Jin, Yijia Xiao, Wenke Lee, B. Aditya Prakash, Haohan Wang
SourceLearn helps LLM agents progressively build reusable expertise about trusted information sources instead of repeatedly starting from scratch.
Assaf Ben-Kish, Akarsh Kumar, James Glass, Raja Giryes
Local Support Learning helps large language models learn new skills without overwriting what they already know by activating updates only where they are locally needed.
Haixin Wang, Xiaoxuan Wang, Junkai Zhang, Han Zhang, Renliang Sun, Alexander K Taylor, Yidan Shi, Haoran Deng, Chenguang Wang, Jason Cong, Yizhou Sun, Wei Wang
ACLArena studies how agents can learn new skills over multiple training stages without forgetting old ones, proposing replay and specialized LoRA experts as a practical solution.
Hongyang Du, Lan Yan, Christian Flores, Asim Kadav
A frozen AI designer gets better over time by turning user interactions and successes into reusable, continually refined design procedures.
Alexander Martin, Benjamin Van Durme
Knowledge Pull Requests make continually updating documents more transparent by showing exactly which claims changed, where they belong, and how the final text was revised.
Ankit Goyal, Jaideep Ray
When an AI agent gets a new model, structured memories survive far better than compressed notes or partially migrated retrieval indexes.
Zora Zhiruo Wang, Apurva Gandhi, Rulin Shao, Aspen Chen, Jonas Mueller, Zhiqi Liang, Jett Chen, Michael Ryan, Qianou Ma, Luxi He, Zhoujun Cheng, Andre He, Seungone Kim, Jiayi Geng, Mingqian Zheng, Weiwei Sun, Zheyuan Zhang, Xinran Zhao, Yike Wang, Abe Hou, Liwei Jiang, Pang Wei Koh, Diyi Yang, Graham Neubig, Daniel Fried
The work shows how agents can learn a user’s evolving standards through repeated interaction, becoming substantially better at personalized writing and visual-creation tasks.
Yifan Zhang, Steve Ta, Jasper Zhang, Jichen Feng, Shuzhen Li, Yongxin Zhang, Yifeng Liu, Huizhuo Yuan, Mengdi Wang, Quanquan Gu, Andrew Chi-Chih Yao
This paper turns attention into an online learning memory, showing how fast-weight updates can help sequence models retain and use information over long contexts.
Qian Kou, Xiaofeng Shi, Xiaosong Qiu, Hua Zhou
IAR teaches language models to memorize and use a document collection without retrieval while preserving their broader abilities.
Maximilian Du, Zhanyi Sun, Chen Xu, Paarth Shah, Masha Itkina, Shuran Song
A small set of carefully chosen past experiences can anchor robot skills and dramatically reduce forgetting while learning new tasks.
James Hartley, Zeropy Surio, Daniel Whitmore, Hannah Clarke, Thomas Reed
SPARCL reduces forgetting in exemplar-free continual learning by protecting important spectral directions while updating only the residual feature space.
Haoqin Tu, Yunhao Fang, Yizhong Wang, Cihang Xie, Shen Yan
Chain-of-Experience helps LLMs learn from repeated feedback during inference, improving performance and token efficiency across math, coding, and knowledge tasks.
Borui Kang, Jinrui Gu, Junhan Lv, Wenbin Li, Lei Wang, Yang Gao
This paper proposes letting frozen AI models keep learning through evolving prompts, memories, tools, and routing rules while guarding against the loss of earlier skills.
Tianci Liu, Zihan Dong, Tianchun Li, Yi-Chung Chen, Qiming Cao, Xingchen Wang, Shiyang Wang, Zichen Miao, Linjun Zhang, Haoyu Wang, Jing Gao
HPSE helps language models turn newly injected passages into usable, multi-step knowledge rather than merely memorized text.
Tianyi Guan, Yiding Wang, Haotong Yang, Siyuan Cao, Shirui Liu, Yi Hu, Jiaqi Li, Muhan Zhang
ContinualSkillBench tests whether LLM agents truly learn reusable skills over time, finding that they often adapt to context without reliably consolidating transferable abilities.
Zixuan Wang, Yuhong Chen, Yuxuan Zhu, Guidong Lei, Zhiluohan Guo, Yu Zhao, Kun Wang, Bangyang Hong, Kangle Wu, Yabo Ni, Anxiang Zeng, Cong Fu, Hui Li
This work keeps recommender systems up to date by separating reusable behavioral knowledge from task-specific adaptation, delivering strong benchmark and real-world business gains.
Alberto Acedo
Omega-S is a cheap weight-only regularizer that helps LLMs remember old skills while learning new ones, though it still needs broader testing.
Sietse Schelpe
Instead of making the model smarter, this system remembers verified answers so it can return them instantly, exactly, and forever.
Jiarui Wang, Xiang Shi, Jiaqi Cao, Rubin Wei, Xiquan Wang, Hao Sun, Jingzhi Wang, Zhiqi Yang, Qipeng Guo, Bowen Zhou, Zhouhan Lin
MemSFT gives LLMs specialized domain expertise through an external memory while preserving their general abilities.
Zeyu Zhang, Ziliang Guo, Yihang Sun, Xichong Zhang, Xixuan Hao, Zehao Lin, Yang Zhang, Xiaoyan Zhao, Tong Shen, Bo Tang, Zhi-Qin John Xu, Junchi Yan, Haofen Wang, Xu Chen, Feiyu Xiong, Zhiyu Li, Tat-Seng Chua
Metis gives foundation models an internal, continually updating memory that can store and recall experience without changing the model’s weights.
Ali Behrouz, Farnoosh Hashemi, Vahab Mirrokni
This paper argues that language models should 'sleep' to consolidate memories and generate synthetic practice, helping them keep learning over time without forgetting.
Zhenlin Hu, Yan Wang, Zhen Bi, Zihao Xue, Bingyu Zhu, Longtao Huang, Xiongtao Zhang, Zeyu Yang, Zhixuan Chu, Jungang Lou
This paper asks whether LLMs can get better at making synthetic data over time by learning from a stream of past synthesis tasks instead of treating each one in isolation.
Pilchen Hippolyte, Fabre Romain, Signe Talla Franck, Perez Patrick, Grave Edouard
This paper shows that the order of pre-training data matters: training large language models on time-ordered text can make them better at knowing which facts were true when, without hurting general language ability.