Explore methods and evaluations for mathematical, logical, and multi-step reasoning. Compare gains against inference cost and benchmark limitations.
39 papers · Latest edition October 5, 2026
Three of the latest briefs in this collection. Read the evidence and the original papers alongside them.
FlyBy teaches small reasoning models to recognize when more internal thinking will not help and instead ask a stronger model for missing knowledge.
RLVR can make reasoning models increasingly use strange internal languages, and preventing that drift may require sacrificing some performance.
REST teaches latent LLM agents to form more causal, minimal, separable, and stable internal thoughts, improving reasoning accuracy and interpretability.
Newest editions first.
Chanuk Lee, Minki Kang, Sangwoo Park, Woongyeong Yeo, Jinheon Baek, Sung Ju Hwang
FlyBy teaches small reasoning models to recognize when more internal thinking will not help and instead ask a stronger model for missing knowledge.
Michael Sullivan, Alexander Koller
RLVR can make reasoning models increasingly use strange internal languages, and preventing that drift may require sacrificing some performance.
Fahd Seddik, Fatemeh Fard
REST teaches latent LLM agents to form more causal, minimal, separable, and stable internal thoughts, improving reasoning accuracy and interpretability.
Shuvendu K. Lahiri
NFV uses AI agents to translate ordinary code into machine-checkable formal proofs, making software verification more accessible while revealing the limits of end-to-end soundness.
Simon Schug, Brenden M. Lake
Reasoning models can solve individual rule tasks yet often fail when the same underlying logic is rearranged, suggesting their thinking is less systematic than it appears.
Felipe Ocampo Osorio, Sebastián Andrés Cajas Ordoñez, Maximin Lange, Rafi Al Attrach, Sahil Kapadia, Zakaria Laouabdia Sellami, Angelo Antonio Talio, Leo Anthony Celi
Instead of trusting language models to do clinical math, the system has them write code for a restricted executor—and shows that this helps larger models but is no universal cure for medical calculation errors.
Ivan Moshkov, Stephen Ge, George Armstrong, Wei Du, Sadegh Mahdavi, Igor Gitman
An open Nemotron-based system uses training and test-time search to generate, check, and refine mathematical proofs, reaching the IMO gold-medal threshold.
Jeffrey Lai, Anthony Bao, John Quinn, William Gilpin
The study argues that when AI reasoning gets harder, models become temporarily chaotic and get trapped near almost-correct solutions.
Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom, Nicholas M. Boffi, İsmail İlkan Ceylan, Jinwoo Kim
Looped Flows lets models spend more computation at inference time by repeatedly refining noisy predictions, achieving strong results on several challenging reasoning benchmarks.
Kevin Du, Alexander Hoyle, Laura Ruis, Acyr Locatelli
This study finds that chain-of-thought text can hint at which steps matter, but remains far from a reliable explanation of how models actually reason.
R. Thomas McCoy, Paul Soulos, Tal Linzen, Paul Smolensky
The paper argues that neural networks, including LLMs, may secretly organize their continuous representations according to discoverable symbolic structures.
Yunpeng Ba, Zhi Zheng, Yue Xie, Jiaqing Li, Xialiang Tong, Tao Zhong, Mingxuan Yuan, Zhichao Lu, Xuyang Wu, Zhenkun Wang
This work argues that Evolution Strategies can preserve broader reasoning diversity than GRPO, improving LLM Pass@K performance while complementing GRPO's strength in Pass@1.
Henry Robbins, Connor Lawless, Madeleine Udell, Ellen Vitercik
FLARE uses LLMs and Lean proofs to reliably check whether automatically generated optimization formulations are mathematically correct.
Shang Wu, Catarina G Belem, Shuyuan Fu, Mark Steyvers, Padhraic Smyth
This study finds that relying more on AI for logic puzzles may boost short-term performance while weakening the development and retention of independent reasoning skills.
Jialong Liu, Yuling Shi, Ning Yang, Xiaodong Gu, Zuchao Li
SRPO teaches language models to turn their own mistakes into step-by-step learning signals, improving long-horizon reasoning and agent performance with relatively little training.
Cheng Qian, Wenting Zhao, Liangwei Yang, Heng Wang, Jielin Qiu, Heng Ji, Silvio Savarese, Huan Wang, Shelby Heinecke
A powerful model can effectively teach a weaker one new problem-solving habits at test time by building a smart software harness around it.
Jean de Dieu Nyandwi, Leena Mathur, Yonatan Bisk, Robert Hawkins, Graham Neubig
The paper finds that thinking models often amplify the appearance of careful reasoning more than the behaviors that truly signal correct answers.
Lushi Pu, Weiming Zhang, Xinheng Xie, Zixuan Fu, Bingxiang He, Hengyu Zhao, Hongya Lyu, Xin Li, Jie Zhou, Yudong Wang
MathForm helps language models turn informal mathematics into reliable Lean proofs by retrieving library knowledge and repeatedly refining outputs against formal verification feedback.
Jinhe Bi, Chennan Zhou, Zengjie Jin, Aniri, Shuo Lu, Wenke Huang, Hu Cao, Xun Xiao, Zhihong Zhu, Volker Tresp, Fei Shen, Yunpu Ma, Tat-Seng Chua
ReflectRL turns expert mistakes into useful training signals by teaching language models to reflect on flawed solutions before learning to solve problems directly.
Mohsen Hariri, Weicong Chen, Nahal Shahini, Vikash Singh, Kai Ye, Amirhossein Samandar, Debargha Ganguly, Sreehari Sankar, Yanyan Zhang, Shouren Wang, Jerry Peng, Biyao Zhang, Michael Hinczewski, Vipin Chaudhary
This paper explains how to fairly compare reasoning LLMs that spend more computation at inference time and provides practical standards for evaluating and reproducing them.
Omatharv Bharat Vaidya, Connor Thomas Jerzak, Zayne Rea Sprague, Fangcong Yin, Nhat Ho
Instead of trusting the most common LLM answer, CALVER checks whether each causal explanation actually obeys the rules of causal graphs.
Satyam Kumar, Saurabh Jha
CADENCE helps small language models learn stronger mathematical reasoning from larger teachers using adaptive distillation, richer rewards, and modest hardware.
Randhir Kumar
Per-chunk fact checking breaks multi-hop RAG because no single passage contains the whole answer, but verifying against decomposed sub-questions substantially restores reliability.
Zheng Wu, Chenhao Xue, Shijie Zheng, Yijie Lu, Cheng Yang, Zhuosheng Zhang
This paper shows that LLMs often know the commonsense answer but get distracted by overly salient details, and that better prompting can help them recover it.
Ifueko Igbinedion, Jillian Ross, Etienne Ricardez, Sertac Karaman, Eric So
Some reasoning models give up not because they lack the ability to solve a problem, but because they mistakenly think they will run out of context before finishing.
Pulkit Madan, Sanjay Haresh, Reza Ebrahimi, Sunny Panchal, Apratim Bhattacharyya, Roland Memisevic
This study finds that vision models relying on local, sequential glimpses can generalize better to longer and more complex visual reasoning tasks than models using global shortcuts.
Rajiv Shailesh Chitale, Rahul Madhavan, Taneesh Gupta, Deepanway Ghosal, Aravindan Raghuveer
TTEL makes language-model reasoning more efficient by finding where a solution went wrong and branching from the last correct step instead of starting over.
Sangmook Lee, Minbeom Kim, Jeonghye Kim, Dohyung Kim, Sojeong Rhee, Kyomin Jung
This paper asks whether LLMs are truly solving math problems in different ways, or just wording the same strategy differently, and finds that many diversity metrics miss the real story.
Ying Fan, Anej Svete, Kangwook Lee
LOTUS uses recurrent Transformer computation to make hidden-state reasoning faster while retaining much of the power and interpretability of explicit chain-of-thought.
Raphaël Sarfati, Pratyush Ranjan Tiwari, Siddharth Boppana, Christopher J. Earls, Srikar Varadaraj, Eric Ho
The study shows that an LLM's hidden representations can reveal its confidence and evidence use more reliably than its spoken chain of thought.
Varun Reddy Nalagatla
The paper argues that transformer hidden states can be read directly in the default coordinate basis, with each dimension acting like a binary semantic register whose sign patterns reveal and even control concepts without training a probe.
Linas Nasvytis, Simon Jerome Han, Ben Prystawski, Satchel Grant, Noah D. Goodman, Judith E. Fan
CORE helps language models get better at reasoning faster by turning comparisons between right and wrong attempts into compact, human-readable strategy notes.
Joshua Ong Jun Leang, Zheng Zhao, Mihaela Cătălina Stoian, Qiyuan Xu, Haonan Li, Wenda Li, Shay B. Cohen, Eleonora Giunchiglia
This paper makes Lean theorem proving cheaper and stronger by combining curriculum training, proof-trace filtering, and a novel data augmentation scheme, while also testing a diffusion-style prover.
Harsh Patel
This paper argues that some reasoning problems can be solved by search but still cannot be learned as a neat left-to-right chain of thought—what models can distill is often verification and memorization, not the search itself.
Mingzhong Sun, Teresa Yeo, Armando Solar-Lezama, Tan Zhi-Xuan
This paper shows that large reasoning models can often solve problems yet still be surprisingly bad at judging whether a step-by-step explanation is actually valid.
Jiacheng Chen, Xinyu Zhang, Shunkai Zhang, Yanmohan Wang, Lin Li, Tiancheng Qin, Qin Wang, Zhengmao Zhu, Tianle Li, Jingyang Li, Zehan Li, Binyang Jiang, Jin Zhu, Han Ding, Fei Yu, Chenyu Du, Zijian Song, Jiayuan Song, Zhi Zhang, Yunan Huang, Weiyu Cheng, Pengyu Zhao, Yu Cheng
MaxProof uses a population of proof candidates plus generator-verifier-ranker loops to push an AI system to gold-medal-level performance on hard math olympiad proofs.
Guowei Xu, Zhenting Qi, Huangyuan Su, Weirui Ye, Himabindu Lakkaraju, Sham M. Kakade, Yilun Du
This paper introduces a new way for language models to improve themselves by both evolving candidate answers forward and breaking problems into checkable subgoals backward, boosting performance on hard reasoning tasks.
Chen Henry Wu, Aditi Raghunathan
This paper shows that teaching a model to verify its own answers better can improve both test-time reasoning loops and training-time self-improvement, leading to stronger performance on hard reasoning tasks.
Ishita Darade, Sushrut Thorat
This paper shows that a Transformer can represent the right intermediate arithmetic steps without actually using them in the way probes suggest, highlighting the gap between what models encode and what they compute.