Memory Anchors for Continual Robot Learning
AuthorsMaximilian Du, Zhanyi Sun, Chen Xu, Paarth Shah, Masha Itkina, Shuran Song
Resources
A small set of carefully chosen past experiences can anchor robot skills and dramatically reduce forgetting while learning new tasks.
Key results
Excluding the top 10% of Memory Anchors increased catastrophic forgetting by 4.5-fold.
A NCHORER reduced forgetting on high-conflict task pairs by 63%.
Anchor-enriched replay reduced average Negative Backward Transfer on LIBERO-Goal by 37%.
A NCHORER achieved 1.7-fold higher final success than random replay on OpenJar and SweaterFold.
What the paper found
Memory Anchors for Continual Robot Learning identifies why ordinary Experience Replay can preserve some robotic skills while catastrophically overwriting others. The critical cases occur when new-task observations overlap with old-task representations but demand conflicting actions, such as opening visually similar jars clockwise versus counterclockwise. The proposed A NCHORER method first detects representation overlap in the policy latent space, then isolates new samples with high action disagreement using noise-conditioned denoising, and finally retrieves the most similar old-task transitions as Memory Anchors for replay. On the LIBERO benchmark, excluding only the top 10% of Memory Anchors caused a 4.5-fold increase in catastrophic forgetting, while reserving 10% of the replay buffer for anchors reduced forgetting on high-conflict task pairs by 63% and reduced average Negative Backward Transfer on homogeneous LIBERO-Goal tasks by 37%. The approach also matters for pretrained vision-language-action models: π0.5 showed the same sensitivity when replay buffers were small. On two real-robot suites, OpenJar and SweaterFold, A NCHORER achieved 1.7-fold higher final success than random replay at the same buffer budget. The findings suggest that continual robot learning depends less on uniformly replaying past data than on retaining a small, targeted set of decision points where visual similarity and action conflict intersect.
Original abstract
Robot policies deployed in the wild should have the capability to continually learn new tasks without forgetting existing behaviors. A common approach to combat such catastrophic forgetting is to train on new task data with a replay buffer of previously learned task data. Although this buffer is commonly sampled randomly from all prior experiences, we show that a small set of these experiences contributes greatly in anchoring past performance. We call these experiences Memory Anchors. We identify Memory Anchors in regions where representations of new-task observations collapse onto those of old-task observations even though the tasks require conflicting actions, like when a familiar object must be manipulated in a new way. Rehearsing old data in this region plays a key role in preventing destructive overwriting of past task knowledge, serving as this critical Memory Anchor role. Excluding only 10% Memory Anchors before sampling the buffer leads to more than a 4.5x increase in catastrophic forgetting on the LIBERO benchmark suites. Conversely, enriching the replay buffer with Memory Anchors can decrease high-conflict task forgetting by 63% and enables successful continual learning of two task sequences on a real robot. Videos and additional visualizations can be found at https://robot-adaptation.github.io/MemoryAnchors
Read the original paperMore in Continual Learning
Browse all 24 papers →ASCENT: Online Test-Time Training of Long-Horizon Agents via Self-Distillation of Verified Experience
Haodong Lu, Dong Gong
ASCENT lets deployed LLM agents learn from verified successes on the fly by converting hindsight about their own trajectories into lasting weight updates.
From Knowledge Access to Source Learning: Developing Source-Specific Competence
Lucheng Fu, Kejing Xia, Yiyang Wang, Yiqiao Jin, Jinjin He, Xiyuan Yang, Haoxin Liu, Ye Yu, Haibo Jin, Yijia Xiao, Wenke Lee, B. Aditya Prakash, Haohan Wang
SourceLearn helps LLM agents progressively build reusable expertise about trusted information sources instead of repeatedly starting from scratch.
Local Support Learning
Assaf Ben-Kish, Akarsh Kumar, James Glass, Raja Giryes
Local Support Learning helps large language models learn new skills without overwriting what they already know by activating updates only where they are locally needed.