Explore how language models are trained, evaluated, and adapted for useful tasks. Compare reported capabilities, training choices, and limitations.
81 papers · Latest edition October 9, 2026
Three of the latest briefs in this collection. Read the evidence and the original papers alongside them.
By sampling and reshaping expert data before training, this work argues that supervised finetuning can match RL while generalizing better and forgetting less.
Language models may repeatedly switch between shallow memorization and genuine reasoning during training, and the paper shows how to detect and potentially control these swings.
The study finds that expensive multi-teacher on-policy distillation may offer little advantage over carefully tuned, cheaper alternatives such as SFT and weight merging.
Newest editions first.
Aayush Karan, Sitan Chen, Yilun Du
By sampling and reshaping expert data before training, this work argues that supervised finetuning can match RL while generalizing better and forgetting less.
Jiaxin Wen, Zhengxuan Wu, Dawn Song, Lijie Chen
Language models may repeatedly switch between shallow memorization and genuine reasoning during training, and the paper shows how to detect and potentially control these swings.
Roy Xie, Dan Friedman, Feng Nan, Yukun Huang, Zhichao Xu, Chengjiu Zhang, Jun Xu, Manaal Faruqui, Vivek Rathod, Bhuwan Dhingra
The study finds that expensive multi-teacher on-policy distillation may offer little advantage over carefully tuned, cheaper alternatives such as SFT and weight merging.
Shangzhe Li, Yuxiao Yang, Tianrun Yu, Kaixiang Zhao, Xiaoyun Wang, Taylor W. Killian, Weitong Zhang
LSPD uses reinforcement-learning ideas to make LLM policy distillation more sample-efficient while preserving the diversity needed for stronger reasoning.
Markus Frohmann, Mahdiyar Ali Akbar Alavi, Elizabeth Lingg, Navid Rekabsaz
LLM judges may rank candidates equally well yet make different decisions when their order changes, and OC-SFT makes those decisions more consistent.
Zhengyu Fang, Seoyeon Hong, Jie Yang, Muyang Li, Koyoshi Shindo, Brandon Joseph Lwowski, Jing Li
Latent-MOPD teaches one language model to combine several specialists by learning both what they predict and how they represent it.
Mohamed Eltahir, Anas Obayd, Raed Rashid, Abdulrahman Alghamdi, Abdulrahman Mousa, Abdallah Ahmed, Tanveer Hussain, Naeemullah Khan
Periscope lets frozen language models search million-token documents on a single GPU by replacing one enormous read with a smart grid of small probes.
Rui Cai, Wenhui Zhu, Xiwen Chen, Jincheng Cao, Han Yu, Shayan Mohajer Hamidi, Zelin He, Qiyao Ma, Daiwei Chen, Xuanzhao Dong, Yuanda Xu, Jelena Markovic-Voronov, Kayhan Behdin, Zhengze Zhou, Ran He, Alborz Geramifard, Rohit Jain, Zhe Zhao
Smaller models can generate more useful negative examples than students themselves, making preference training for large language models cheaper and potentially better.
Hyungjin Lukas Kim
Most open-source LLM applications migrate only after a provider shuts down their model, revealing a major and measurable dependency risk.
Peter Chen, Xi Chen, Wotao Yin, Tianyi Lin
ComPO aligns language models using preference comparisons rather than direct likelihood optimization, aiming to reduce failures caused by small preference margins.
Gaoyuan Du, Anam Nawaz Khan, Rex Zhou, Xiaoyang Liu, Deepayan Chakrabarti, Fnu Suya, Xueping Li
Even supposedly deterministic greedy LLM decoding can produce different answers in BF16 and FP16, but selectively recomputing the final layer in FP32 can greatly improve agreement at low cost.
Nathanaël Carraz Rakotonirina, Momchil Hardalov, Gonzalo Iglesias, Adrià de Gispert
This study asks whether new knowledge belongs in an LLM’s context, hidden representations, or weights, finding that KV-cache-based Cartridges often deliver the best accuracy but can cause forgetting.
Jacqueline He, Howard Yen, Shuyue Stella Li, Margaret Li, Hanqing Zeng, Yinglong Xia, Benyu Zhang, Zhuokai Zhao, Qiang Zhang, Pang Wei Koh, Luke Zettlemoyer, Wen-tau Yih
A new distillation strategy uses teacher confidence to help language models gain reasoning skills without sacrificing the factual knowledge they learn during mid-training.
Zixuan Fu, Bingxiang He, Yuxin Zuo, Haohuan Huang, Jinqian Zhang, Ruhang Xiao, Cheng Qian, Qinyu Luo, Huan-ang Gao, Yudong Wang, Zhiyuan Liu, Ning Ding, Chaojun Xiao
This study finds that on-policy distillation of language models may need far fewer examples than expected, but many more training steps to fully absorb the supervision.
Ke Yang, Chenglong Wang, Michel Galley, Chandan Singh, Jeevana Priya Inala, ChengXiang Zhai, Jianfeng Gao
StudentSim teaches LLMs to imitate individual learners and respond realistically to tutoring, enabling more personalized AI tutors.
Katherine Van Koevering, Anjalie Field
The study finds that everyday linguistic styles associated with women can cause LLMs to produce shorter and less formal workplace writing, revealing a subtle but consequential form of bias.
Davide Romano, Kanak Raj, Jerrod Parker, Daniele Giofrè
Test-time scaling can generate better answers, but for open-ended tasks the real challenge is reliably choosing or synthesizing the best one.
Ziyang Luo, Zhongyao Chu, Xinjie He, Youting Wang, Xukui Qin, Runxiong Wu, Yan-Syuan Chen
YOPO lets a frozen language model reason, steer itself, and abstain when uncertain—all in one forward pass.
Ananya Sahu, Mohit Bansal, Elias Stengel-Eskin
CreativeInstruct teaches LLMs to produce more diverse and creative outputs without sacrificing quality, and shows that this creativity can also improve downstream reinforcement learning.
Tao Feng, Fangxu Yu, Haozhen Zhang, Zhongjie Dai, Liangqi Yuan, Zijie Lei, Weizhi Zhang, Kunlun Zhu, Haodong Yue, Keyang Xuan, Ge Liu, Jiaxuan You
LLMRouter is an open platform and benchmark for choosing the best language model for each query while balancing answer quality, personalization, and inference cost.
Chengyu Lai, Jiuning Lin, Zhibo Xiao, Xiaodong Zhu, Ruiquan Lan, Bin Zhang, Zihong Huang, Wendong Zhang, Chuxin Chen, Yinjiang Cai, Shuai Zhong, Lingqing Zhang, Dimin Wang, Jialin Zhu, Han Zhu
MetaStrategy uses a compact LLM to generate safe, executable ranking policies that improve real-world recommendation outcomes without adding online latency.
Kazuki Nakayashiki, Keisuke Watanabe
Language models often agree with one another far more than they agree with real readers, suggesting that model consensus should not be mistaken for human consensus.
Sining Zhoubian, Dan Zhang, Evgeny Kharlamov, Jie Tang
This survey explains how modern language models remember, update, and retrieve information, and organizes the field’s many competing memory designs into one coherent framework.
Marta Garnelo, Wojciech M. Czarnecki
LLMs can reason impressively over many modalities, but this study finds that their tabular prediction ability collapses as the number of input dimensions grows.
Beining Wang, Weihang Su, Hongtao Tian, Hao Kong, Tao Yang, Ting Yao, Qingyi Pan, Yueyue Wu, Qingyao Ai, Min Zhang, Yiqun Liu
DynamicRubric keeps LLM evaluators useful as policies improve by generating adaptive rubrics tailored to each set of competing responses.
Guillaume Coqueret, Joan Llull, Florian Oswald, Christophe Pérignon, Christoph Scheuch, Lars Vilhuber
This paper explains why LLM results can change unexpectedly and proposes practical rules for making LLM-based research more reproducible.
Zing Team, Ao Xiang, Bi Jingping, Chen Jiahui, Chen Lehan, Chen Yilin, Cheng Xueqi, Fan Yixing, Gan Kairong, Gao Haowen, Gao Jinhua, Gao Shuxuan, Gong Chang, Guo Jiafeng, Guo Ruijie, Han Zhouyu, He Guangfu, He Yichun, Jiang Shuo, Jing Shaoling, Jing Ya, Lei Chenhao, Lei Yan, Li Anqi, Li Chengao, Li Haoyu, Li Shitian, Liang Xinjian, Liu Zhaoge, Lyu Xingyu, Nie Zhuwei, Pang Liang, Quan Zeping, Shan Shiguang, Shen Huawei, Tang Xinran, Tian Feng, Wang Qian, Wang Ruiping, Wang Xiaohong, Xia Zaiyu, Xiao Yi, Xu Jiayuan, Xu Kehan, Xu Qianqian, Xu Tianyu, Xu Yongjun, Yang Haoming, Yang Jun, Yao Di, Yu Xiaoming, Zhang Futong, Zhang Jie, Zhang Shixuan, Zhang Yuxuan, Zhao Xinyu, Zhao Zhuoran, Zhong Yunfei, Zhu Shengyu
Zing gives LLMs a way to measure, learn, and apply social intelligence through specialized benchmarks, training, and runtime support.
Rui Wang, Hongru Wang, Yi Chen, Boyang Xue, Tianqing Fang, Wenhao Yu, Kam-Fai Wong
This paper explains why on-policy distillation can go wrong in LLM training and shows that regulating its guidance signals can make exploration more reliable.
Chen Wang, Zhaochun Li, Jionghao Bai, Yining Zhang, Hexuan Deng, Ge Lan, Yue Wang
Distilled RL combines reinforcement learning with selective teacher guidance to help LLMs acquire new knowledge more effectively than standard RL or direct logit matching.
Tomas Bruckner
A handful of one-token answers to simple prompts may be enough to tell whether an API is really serving the LLM it claims to use.
Nishant Aggarwal, Ayushi Dubal, Sreeraj Kannakarankodi, Ian McDougall, Adarsh Mittal, Vishnu Ramadas, Noah Scott, Ranganath Selagamsetty, Weichu Yang, Karthikeyan Sankaralingam
This study finds that a carefully structured team of LLM reviewers can often critique computer architecture papers more deeply than individual human analysts, while still struggling with trust and calibration.
Patrik Wolf, Thomas Kleine Buening, Andreas Krause, Celestine Mendler-Dünner
The study shows that language models often know how subgroups differ but fail to combine that knowledge into accurate population-level answers.
Zhilong Zhang, Hongli Yu, Huan-ang Gao, Hanlin Wu, Yuxuan Song, Wei-Ying Ma, Ya-Qin Zhang, Hao Zhou
SAR trims language-model updates down to their most useful spectral directions, aiming to preserve reasoning while improving exploration, capability consolidation, and model merging.
Yifan Niu, Han Xiao, Dongyi Liu, Zelong Wang, Dihong Gong, Yasheng Wang, Jia Li
This paper lets LLM distillation work across different tokenizers, making it easier and cheaper to transfer knowledge between otherwise incompatible model families.
Xingyu Su, Jacob Helwig, Shubham Parashar, Atharv Chagi, Lakshmi Jotsna, Degui Zhi, James Caverlee, Dileep Kalathil, Shuiwang Ji
This paper shows how to turn an autoregressive language model into a diffusion-style language model much more efficiently by training it on its own generated trajectories while distilling knowledge from the original model.
Giovanni Dettori, Matteo Boffa, Danilo Giordano, Idilio Drago, Marco Mellia
The paper shows that LLMs struggle not just with long inputs, but especially with long inputs packed with lots of new information, sharply reducing their effective context window.
Phuong Tuan Dat, Qi Li, Xinchao Wang
dOPSD teaches diffusion language models using their own unfolding predictions, boosting reasoning without needing ground-truth hints.
Chenhao Dang, Jing Ma, Mingjie Liao
This paper uses reinforcement learning to automatically decide what training data an LLM should see next, making pre-training faster and improving final model quality.
Zhennan Shen, Yanshu Li, Qingyu Yin, Chak Tou Leong, Zhilin Wang, Yanxu Chen, Rongduo Han, Sunbowen Lee, Yi R. Fung
This paper shows that on-policy distillation for language models follows its own unique path in parameter space, with updates quickly locking into a low-dimensional subspace that still preserves performance.
Etienne Casanova, Rafal Kocielnik, R. Michael Alvarez
The paper shows that LLMs can be stubborn annotators: once they form a confident answer, extra prompting often fails to change their mind, and what matters most is whether the model’s internal concept matches the task definition.
Chensong Huang, Changyu Chen, Chenwei Lin, Hanjia Lyu, Xian Xu, Jiebo Luo
This paper shows that LLMs can look like humans in risk-taking tasks while actually using very different decision mechanisms underneath.
Yong Yi Bay, Kathleen A. Yearick
The paper argues that beyond a modest number of samples, test-time scaling for language models stops helping and can even hurt because the real bottleneck is picking the right answer, not generating more candidates.
Xiang Hu, Xinyu Wei, Hao Gu, Minshen Zhang, Tian Liang, Huayang Li, Lei Zhu, Yan Wang, Sirui Han, Yushi Bai, Kewei Tu, Haitao Mi, Leo Liang
HiLS Attention is a new way to make LLMs handle much longer contexts by learning which chunks to attend to, cutting compute while keeping performance strong.
Wenhan Ma, Jianyu Wei, Liang Zhao, Hailin Zhang, Bangjun Xiao, Lei Li, Qibin Yang, Bofei Gao, Yudong Wang, Rang Li, Jinhao Dong, Zhifang Sui, Fuli Luo
MOPD is a new way to merge multiple specialized LLM skills by distilling several RL-trained teachers into one student using the student’s own rollouts, and it appears to work well at frontier scale.
Yonggan Fu, Lexington Whalen, Abhinav Garg, Chengyue Wu, Maksim Khadkevich, Nicolai Oswald, Enze Xie, Daniel Egert, Sharath Turuvekere Sreenivas, Shizhe Diao, Chenhan Yu, Ye Yu, Weijia Chen, Sajad Norouzi, Jingyu Liu, Shiyi Lan, Ligeng Zhu, Jin Wang, Jindong Jiang, Morteza Mardani, Mehran Maghoumi, Song Han, Ante Jukić, Nima Tajbakhsh, Jan Kautz, Pavlo Molchanov
This paper introduces a language model that can switch between autoregressive, diffusion, and self-speculative decoding to trade off speed and quality, achieving major throughput gains over current open-source models.
Zinan Tang, Yukun Zhang, Shaomian Zheng, Zhuoshi Pan, Qizhi Pei, Dingnan Jin, Jun Zhou, Yujun Wang, Biqing Huang
CausalMix treats LLM data mixing like a causal problem, using learned treatment effects to choose better training mixtures that scale to larger models and data pools.
Ali Asaria, Tony Salomone, Deep Gandhi
This paper shows that a diffusion language model’s token order is not truly parallel or sequential, and that how you measure it can completely change the story.
Reza Bayat, Ali Behrouz, Aaron Courville
This paper argues that language models should spend more capacity in earlier layers and less in later ones, and shows that a tapered width schedule can improve performance without increasing cost.
Wonjun Kang, Kevin Galim, Seunghyuk Oh, Minjun Kang, Sanghyun Park, Donghoon Kim, Minjae Lee, Minseo Kim, Rishabh Tiwari, Yuchen Zeng, Hyung Il Koo, Kangwook Lee
AsyncOPD shows how to train language models faster with asynchronous on-policy distillation without losing accuracy, by carefully handling stale rollouts and teacher-feedback estimates.
Jian Yang, Shawn Guo, Wei Zhang, Tianyu Zheng, Yaxin Du, Haau-Sing Li, Jiajun Wu, Yue Song, Yan Xing, Qingsong Cai, Zelong Huang, Chuan Hao, Ran Tao, Xianglong Liu, Wayne Xin Zhao, Mingjie Tang, Weifeng Lv, Ming Zhou, Bryan Dai
LoopCoder-v2 shows that, for parallel loop transformers, two loops are the sweet spot: enough extra computation to boost code and agent performance, but not so much that positional mismatch overwhelms the gains.
Rajat Ghosh, Datta Nimmaturi, Aryan Singhal, Vaishnavi Bhargava, Henry Wong, Johnu George, Debojyoti Dutta
This paper turns GRPO training into a predictable dynamical system, giving a closed-form explanation for reward growth, stability, and failure modes in LLM reasoning training.
Jiayu Yang, Chao Chen, Shengen Wu, Yinhong Liu, Yuxuan Fan, Lujundong Li, Songning Lai, Chengwei Qin, Zhijiang Guo
This paper introduces SWITCH, a way to turn latent reasoning on and off with special tokens so large language models can be trained with reinforcement learning while keeping their hidden reasoning more interpretable.
Shen Nie, Qiyang Min, Shaoxuan Xu, Zihao Huang, Yuxuan Song, Yong Shan, Yankai Lin, Wayne Xin Zhao, Chongxuan Li, Ji-Rong Wen
This work shows that large language models can be trained with masked diffusion instead of next-token prediction and still become highly competitive on reasoning, math, and coding tasks.
Chenrui Fan, Yize Cheng, Ming Li, Soheil Feizi, Tianyi Zhou
This paper shows that you can predict whether an LLM steering trick will work by looking at its early hidden states, then use that prediction to search for better steering settings much more cheaply.
Xuanming Zhang, Sining Zhoubian, Yuxuan Chen, Tianyi Tang, An Yang, Sean Du, Chujie Zheng, Fei Huang, Dayiheng Liu, Gao Huang, Jingren Zhou
This paper shows that for some aligned LLMs, the best next-token predictions may come from a near-final layer rather than the last one, and it uses that insight to improve reasoning with almost no extra cost.
Nirit Nussbaum-Hoffer, Nitay Calderon, Liat Ein-Dor, Roi Reichart
This paper turns LLM decision-making into a causal graph of human-interpretable concepts, using counterfactual chains to make the model’s reasoning more transparent.
Gabriel Freedman, Francesca Toni
This paper shows that LLMs can make structured choices for hidden reasons that their own explanations only partly reveal.
Lechen Zhang, Jiarui Liu, Tal August
This paper shows that LLM personalization looks much better on synthetic tests than on real human conversations, and introduces human-grounded data and evaluation methods to expose the gap.
Yuanxin Liu, Ruida Zhou, Xinyan Zhao, Amr Sharaf, Hongzhou Lin, Arijit Biswas, Mohammad Ghavamzadeh, Zhaoran Wang, Mingyi Hong
REVES improves LLM test-time reasoning by learning from near-miss answers, teaching models not just to solve problems but to recover from mistakes more effectively.
Haipeng Luo, Qingfeng Sun, Songli Wu, Can Xu, Wenfeng Deng, Han Hu, Yansong Tang
STARE is a new training trick for LLM reasoning that keeps exploration alive by reweighting the tokens most responsible for entropy collapse during reinforcement learning.
Mario Sanz-Guerrero, Manuel Mager, Katharina von der Wense
This paper shows that LLMs are strangely more confident in answers they think are their own, and that a simple prompt trick can make them better calibrated without retraining.
Yuhang Zhou, Lizhu Zhang, Yifan Wu, Mingyi Wang, Peng Bo, Jiayi Liu, Xiangjun Fan, Zhuokai Zhao
OmniOPD is a new way to train smaller models from stronger teachers without needing token logits, using chunk-level semantic verification and uncertainty-based checks to improve distillation, especially on math tasks.
XiuYu Zhang, Yi Shan, Junfeng Fang, Zhenkai Liang
This paper shows that base LLMs already have hidden self-evaluation skills and that a small amount of targeted training can surface them to better predict how external judges will score their answers.
Zhou Ziheng, Jiaqi Li, Huacong Tang, Ying Nian Wu, Demetri Terzopoulos
This work shows that for on-policy distillation, letting the student only roll out the first few response tokens can actually outperform full rollouts, while being faster and more stable to train.
Jason Miklian, John E. Katsos
This study shows that online accusations of “AI slop” are growing fast, but they often function more as social gatekeeping than as real detection of LLM-generated writing.
Li Jiang, Haoran Xu, Yichuan Ding, Amy Zhang
This paper improves LLM distillation by fixing bad reasoning prefixes at the trajectory level, helping smaller models learn better from teacher-guided rollouts.
Tao Chen, Gangwei Jiang, Pengyu Cheng, Siyuan Huang, Yihao Liu, Jingwei Ni, Jiaqi Guo, Mengyu Zhou, Kai Tang, Junling Liu, Qinliang Su, Xiaoxi Jiang, Guanjun Jiang
Skill-RM turns reward evaluation into an agent-like skill that dynamically combines rules, references, and rubrics to judge LLM outputs more consistently.
Arda Uzunoglu, Alvin Zhang, Daniel Khashabi
This paper learns when to trust weak labels so a stronger model can turn imperfect supervision into nearly lossless training and even bootstrap itself repeatedly.
Yaoming Li, Guangxiang Zhao, Qilong Shi, Lin Sun, Xiangzheng Zhang, Tong Yang
A comprehensive primer that maps the emerging landscape of post-training reasoning data and explains what makes it work for modern large language models.
Yu Xia, Zhouhang Xie, Xin Xu, Byungkyu Kang, Prarit Lamba, Xiang Gao, Julian McAuley
This paper teaches an LLM to think more efficiently by steering its reasoning step-by-step with a controller that balances accuracy and token budget.
Xu Wan, Speed Zhu, Jianwei Cai, Guang Chen, XiMing Huang, Wiggin Zhou, Mingyang Sun
This paper treats LLM reasoning like an economy, using shadow prices to decide which queries deserve more compute so limited budget yields the best overall accuracy.
John Seon Keun Yi, Aaron Mueller, Dokyun Lee
This paper shows how to compress expensive multi-agent debate into one language model, making reasoning cheaper and exposing interpretable internal agent-like directions that can also be steered for safety control.
Zhihao Wu, Gracia Gong, Qinglin Zhu, Yudong Chen, Runcong Zhao
This paper shows that averaging outputs from a few LLMs can wash away text watermarks, exposing a major weakness in current AI-generated text detection methods.
Mingze Wang, Shuchen Zhu, Yuxin Fang, Binghui Li, Kai Shen, Shu Zhong
This paper shows that the tiny scale vectors inside LLM normalization layers matter a lot for training, and proposes simple changes that make large language models learn better with almost no extra cost.
Yangyi Huang, Ruotian Peng, Zeju Qiu, Jiale Kang, Yandong Wen, Bernhard Schölkopf, Weiyang Liu
PEFT-Arena reframes LLM finetuning as a balance between learning new tasks and preserving old abilities, revealing which parameter-efficient methods best avoid forgetting.
Xu Ouyang, Deyi Liu, Yuhang Cai, Jing Liu, Yuan Yang, Chen Zheng, Thomas Hartvigsen, Yiyuan Ma
This paper treats LLM training like information flowing through a noisy channel, offering a new scaling law that explains why bigger models and more data can sometimes make performance worse.
Maxence Lasbordes, Amélie Chatelain, Djamé Seddah
This paper shows that multilingual reasoning in LLMs can be improved by swapping in stronger mid-layers from an English model, revealing a shared reasoning core beneath language-specific outer layers.
Reese Levine, Rithik Sharma, Nikhil Jain, Abhijit Ramesh, Zheyuan Chen, Neha Abbas, James Contini, Tyler Sorensen
This paper shows how to run large language models efficiently in the browser with WebGPU, cutting memory use and boosting speed across many devices and quantization formats.
Dongxin Guo, Jikun Wu, Siu Ming Yiu
This paper argues that model collapse in LLMs is really a kind of cultural evolution problem, and shows that self-training can first improve then damage compositional language structure before eventually degrading it.
Victoria Lin, Taedong Yun, Maja Matarić, John Canny, Arthur Gretton, Alexander D'Amour
This paper argues that many LLM simulation experiments are really observational studies in disguise, and it proposes causal tools to detect and reduce the hidden bias from shifting synthetic users.
Di He, Songjun Tu, Keyu Wang, Lu Yin, Shiwei Liu
This paper uses heavy-tail theory to automatically give different Transformer layers different learning rates, helping large language models train faster and perform better.