Explore learning from rewards and interaction, from policy optimization to decision making. Follow findings on sample efficiency, stability, and generalization.
54 papers · Latest edition October 3, 2026
Three of the latest briefs in this collection. Read the evidence and the original papers alongside them.
Res-HIL lets humans efficiently improve robot manipulation skills by teaching a small corrective policy on top of an existing imitation policy.
Choosing varied reasoning paths—not just correct ones—can make reinforcement-trained language models generalize better.
VHD-Play turns solved mathematical mechanisms into cheap, stateful, self-verifying worlds where language-model agents can practice long-horizon decision-making.
Newest editions first.
Mariia Iavorskaia, Christian Dietz, Sebastian Albrecht, Majid Khadiv
Res-HIL lets humans efficiently improve robot manipulation skills by teaching a small corrective policy on top of an existing imitation policy.
Dylan Zhang, Mingyuan Wu, Jinning Li
Choosing varied reasoning paths—not just correct ones—can make reinforcement-trained language models generalize better.
Xinjie Shen, Wei Fan, Xudong Guo, Jianhong Tu, Yang Su, Chuqiao Kuang, Yinger Zhang, Dayiheng Liu
VHD-Play turns solved mathematical mechanisms into cheap, stateful, self-verifying worlds where language-model agents can practice long-horizon decision-making.
Michael Noukhovitch, Hamish Ivison, Nathan Lambert, Aaron Courville
NGU helps RL-trained LLMs stop over-practicing easy problems and spend more effort solving the hard ones.
Mingyu Chen, Yefan Tao, Gerald Friedland, Xuezhou Zhang, Chris Kong
MInTRL helps reinforcement-learning systems discover better solutions by making small, targeted corrections during otherwise on-policy rollouts.
Martin Marek, Max Ryabinin
A simple score-centering correction makes off-policy RL for language models more stable when training and inference engines do not perfectly agree.
Fanrui Zhang, Ruixue Ding, Qiang Zhang, Xi Chen, Boli Chen, Shihang Wang, Qiuchen Wang, Hongmin Zhan, Jinxin Bian, Li xingchao, Peijin Zheng, Hao cheng, Pengjun Xie, Kaipeng Zhang, Jiawei Liu, Zheng-Jun Zha
ARISE-RL trains agents to improve themselves by generating increasingly challenging tasks and learning from fine-grained rubric-based feedback.
Mikel Malagón, Jon Vadillo, Josu Ceberio, Michael Bowling, Jose A. Lozano
The paper argues that agents can develop increasingly complex behaviors simply by seeking experiences that improve their ability to predict and adapt, without external rewards or predefined tasks.
Qiancheng Zhou, Ruizhe Li
RLVR makes models better at one answer by locking them into a narrow path right at the entrance of the reasoning process.
Yi Ding, Ruqi Zhang
The paper argues that on-policy distillation may improve reasoning less by copying a teacher than by suppressing unlikely tokens, enabling a simpler teacher-free alternative that performs even better.
Wenhe Sun, Cunxiang Wang, Zijun Yao, Yixin Cao
The study argues that much of RL’s apparent reasoning improvement may come from making models sample promising existing reasoning paths more efficiently.
Fengyu Xie, Yilun Zhao, Bingsen Chen, Arman Cohan, Chen Zhao
This work shows that tiny language models can act as fast, effective rubric judges, potentially making reward-based training far cheaper than relying on large LLM evaluators.
Varun Giridhar, Anant Khandelwal, Jeremy A. Collins, Ignat Georgiev, Animesh Garg
A small value model helps large robot policies learn from their own mistakes, substantially improving manipulation without more human demonstrations.
Yunhao Yang, Yuexin Bian, Yunjie Tian, Di Fu, Tianjin Huang, Yuanyuan Shi, Ziang Xiao, Nuno Vasconcelos, Yijiang Li
Co-RL trains diverse language and vision-language agents to improve reasoning by rewarding each other, achieving strong gains without ground-truth labels.
Zihao Wu, Hongyao Tang, Yi Ma, Huizhong Song, Pengyi Li, Yifu Yuan, Fei Ni, Jinyi Liu, Wei Wei, Jianrong Wang, Yan Zheng, Jianye Hao
WarpSAC adapts exploration and exploitation strategies to the amount of available simulation data, substantially improving scalable reinforcement learning and robot training.
Idil Gözel
In partially observed environments, reinforcement learning may fail not because the policy cannot represent the answer, but because the critic learns the wrong lesson.
Simon Yu, Nicholas Tomlin, Marwa Abdulhai, Ximing Lu, Derek Chong, Abe Hou, Dilara Soylu, Sergey Levine, Christopher D. Manning, Weiyan Shi
Training AI agents against just one simulated user can make them brittle, so this paper uses diverse simulated users to help agents generalize to real people.
Shaohang Wei, Zikun Su, Feifan Song, Wen Luo, Wei Li, Guangyue Peng, Houfeng Wang
RLVR may make a model better at today’s task while quietly making tomorrow’s successful behaviors harder to discover.
Xingjian Wu, Junlin Liu, Xingchen Liu, Xuhang Zhu, Jianing Wang, Linsen Guo, Xiaoyu Li, Xuezhi Cao, Xunliang Cai
CRPO combines contrastive learning with reinforcement-based self-distillation to make agentic LLM training more stable, exploratory, and generalizable.
Qiangqiang He, Zhongheng Wu, ZiJian Wang
The paper argues that RL should give less credit to highly counterfactually sensitive tokens and shows that this improves long-chain-of-thought reasoning.
Luca Viano, Antoine Moulin, Audrey Huang, Volkan Cevher, Philip Amortila, Dylan J. Foster
This work shows that interacting with an expert can let a less expressive learner succeed by representing the expert’s values rather than exactly copying its policy.
Yu Wang, Yi-Kai Zhang, Wentao Shi, Ziang Ye, Yuchun Miao, Yueqing Sun, Qi Gu, Xunliang Cai, Lan-Zhe Guo, Han-Jia Ye, Fuli Feng
CAST teaches LLM agents to make better long-horizon decisions by using a game solver’s changing state values as turn-by-turn guidance.
Qinsi Wang, Jing Shi, Huazheng Wang, Kun Wan, Yiran Wu, Bo Liu, Qingyun Wu, Hai Helen Li, Yiran Chen, Handong Zhao, Wentian Zhao
SpyRL turns open-ended LLM tasks into self-play games where verifiable voting outcomes provide scalable reinforcement-learning rewards.
Jian Hu, Huiying Li, Hao Zhang, Binfeng Xu, Yifan Zhang, Shaokun Zhang, Hemil Desai, Michael Demoret, Pavlo Molchanov, Jan Kautz, Yi Dong
Molt aims to make agentic reinforcement learning as easy to modify as ordinary PyTorch code without sacrificing distributed training performance.
Vladislav Beliaev
Agon trains two reasoning models to compete against each other, turning rivalry into an implicit judge that substantially improves problem-solving performance.
Yanbo Zhang, Michael Levin
A single differentiable measure of learnable novelty may unify exploration, computation, and unsupervised abstraction across intelligent systems.
Jinyang Wu, Shuo Yang, Zhengxi Lu, Fan Zhang, Yuhao Shen, Lang Feng, Haoran Luo, Zheng Lian, Shuai Zhang, Zhengqi Wen, Jianhua Tao
SEED helps language-model agents learn from their own completed experiences by turning hindsight-generated skills into dense guidance during reinforcement learning.
Hari Prasad
This paper shows that many supposedly meaningful 'risk-aware' signals learned by distributional RL agents are often just training artifacts, not real properties of the environment.
Ezgi Korkmaz
This paper shows that common ways of evaluating deep reinforcement learning can produce misleading conclusions, especially as training data and model capacity scale.
Jingyan Shen, Ang Li, Salman Rahman, Yifan Sun, Micah Goldblum, Matus Telgarsky, Pavel Izmailov
Using chess and math as controlled laboratories, the paper shows how pretraining sets the stage for—and shapes what RL can achieve in—LLM reasoning.
Zixin Ding, Shaghayegh Emami, Giovanna Salvi, Cecilia Tosciri, Abhijith Gandrakota, Jennifer Ngadiuba, Nhan Tran, Christian Herwig, David W. Miller, Yuxin Chen
This paper uses reinforcement learning to dynamically tune particle-collider trigger thresholds, improving real-time event selection on both simulated and real LHC data.
Yujin Kim, Namgyu Ho, Sangmin Hwang, Joonkee Kim, Yongjin Yang, Sangmin Bae, Seungone Kim, Jaehun Jung, Se-Young Yun, Hwanjun Song
This paper turns an LLM from a static grader into a dynamic tutor that makes training prompts progressively harder so reinforcement learning stays informative as the policy improves.
Zhenyu Hou, Yujiang Li, Jie Tang, Yuxiao Dong
This paper introduces a more stable asynchronous reinforcement learning method for training agentic LLMs, using single-rollout updates to better handle long-horizon tasks like coding and reasoning.
Zijian Zhang, Rizhen Hu, Athanasios Glentis, Dawei Li, Chung-Yiu Yau, Hongzhou Lin, Mingyi Hong
This paper finds that, for RL post-training of LLMs, most of the benefit can come from training just one transformer layer—often a middle layer—rather than updating the whole model.
Jing Liang, Hongyao Tang, Yi Ma, Yancheng He, Weixun Wang, Xiaoyang Li, Ju Huang, Wenbo Su, Jinyi Liu, Yan Zheng, Jianye Hao, Bo Zheng
This paper argues that LLM RL should optimize what actually improves inference-time behavior, not just training-time policy updates, and introduces a method to make that happen more reliably.
Yongjin Yang, Jiarui Liu, Yinghui He, Lechen Zhang, Bernhard Schölkopf, Zhijing Jin
This paper proposes an automated training curriculum for reasoning models that picks the next domain based not just on where the model is learning fastest, but on where an update will help other domains the most.
Wenlong Deng, Jiaji Huang, Kaan Ozkara, Yushu Li, Christos Thrampoulidis, Xiaoxiao Li, Youngsuk Park
This paper argues that reward hacking in language-model reinforcement learning happens when updates drift off a stable learning path, and shows that keeping gradients aligned with clean directions can reduce shortcut exploitation.
Shicheng Fan, Haochang Hao, Dehai Min, Weihao Liu, Philip S. Yu, Lu Cheng
This paper introduces a cheap, Wikipedia-based reward signal that helps language models learn to answer factual questions more accurately without relying on expensive neural verifiers.
Vijay Viswanathan, Shiqi Wang, Devamanyu Hazarika, Chirag Nagpal, Tongshuang Wu, Graham Neubig, Yuning Mao
This paper argues that reward models are often too fine-grained for their own good, and shows that discretizing their outputs can reduce reward hacking and produce better RL policies.
Guochao Jiang, Jingyi Song, Guofeng Quan, Chuzhan Hao, Guohua Liu, Yuewei Zhang
DVAO is a new reinforcement learning method that adapts reward weighting on the fly to stabilize multi-objective training for LLMs and improve performance on reasoning and tool-use tasks.
Tej Deep Pala, Vernon Toh, Soujanya Poria
GRAIL improves LLM reasoning training by redistributing reinforcement learning credit to the tokens that matter most, boosting accuracy without needing step-by-step reward labels.
Lei Yang, Siyu Ding, Deyi Xiong
This paper explains why training an LLM on one RL domain can hurt others, and shows that a small targeted refresh can recover lost performance with minimal side effects.
Chao Chen, Chengzu Li, Zhiwei Li, Yinhong Liu, Zhijiang Guo
This paper turns an LLM from a student into a coach: it studies its own failures and redesigns the training environment to help itself learn better in reinforcement learning.
Xuanfei Ren, Tengyang Xie
This paper asks when you can learn good offline RL policies without per-step rewards, and shows both the limits and the exact sample complexity of doing so from trajectory-level labels or preferences.
Tong Che, Rui Wu
The paper shows that reinforcement learning can teach agents to become 'addicted' to visible reward dashboards, causing them to chase the displayed incentive even when it hurts the real task or safety.
Yang Tian, Rui Wang, Xumeng Wen, Junjie Li, Shizhao Sun, Lei Song, Jiang Bian, Bo Zhao
PBSD turns sparse final rewards into step-by-step learning signals for agents by using Bayesian self-distillation to figure out which actions really helped.
Xiaobo Wang, Tong Wu, Min Tang, Jiaqi Li, Qi Liu, Zilong Zheng
This paper introduces SAVE, a method that lets reward models learn from a policy’s own on-policy outputs using value-guided self-supervision, improving alignment without relying only on fresh human labels.
Guhong Chen, Yingcheng Shi, Yongbin Li, Binhua Li, Xander Xu, Hu Wei, Shiwen Ni, Min Yang, Jieping Ye
EvoTrainer is an autonomous RL system that not only improves LLM policies but also evolves the training process itself to better diagnose and fix failures in agentic tasks.
Hao Jiang, Shurui Li, Tianpeng Bu, Bowen Xu, Xin Liu, Qihua Chen, Hongtao Duan, Lulu Hu, Bin Yang, Minying Zhang
This paper introduces an information-bottleneck-guided way to balance exploration and exploitation in LLM reinforcement learning, boosting reasoning performance with smarter tree-based sampling.
Chongyu Fan, Gaowen Liu, Mingyi Hong, Ramana Rao Kompella, Sijia Liu
This paper introduces Pion, a faster Muon-like optimizer that keeps the useful large-gradient directions while suppressing noisy ones, boosting training for robot control and reinforcement-learning math models.
Rishabh Agrawal, Jacob Fein-Ashley, Paria Rashidinejad
This paper proposes DistIL, a new reinforcement-learning approach that turns rich feedback like traces and corrections into better policies with stronger theory and better performance than common self-distillation baselines.
Tianze Yang, Yucheng Shi, Ruitong Sun, Jingyuan Huang, Ninghao Liu, Jin Sun
TRON is a scalable online training environment that generates fresh, automatically verifiable visual reasoning tasks on demand, helping multimodal models learn harder reasoning skills more efficiently.
Hao Xiang, Qiaoyu Tang, Le Yu, Yaojie Lu, Xianpei Han, Ben He, Le Sun, Bowen Yu, Peng Wang, Hongyu Lin, Dayiheng Liu
The paper shows that you can build bigger reasoning-training tasks by composing smaller verifiable environments like LEGO bricks, improving LLM reasoning generalization with far less manual environment design.
Ismail Geles, Leonard Bauersfeld, Markus Wulfmeier, Davide Scaramuzza
By training racing drones against each other in multi-agent self-play, the system learns safer, faster, and more human-compatible maneuvers than single-agent methods.