All Quiet on the Neural Front: Language Models Do Not Report Injected Pain States
AuthorsEmilio Ferrara
AffiliationsUniversity of Southern California
Models’ internal signals can reveal an injected pain-like state and change their behavior, even when the models themselves do not report feeling it.
Key results
Reports collected across the three 7B and 8B model configurations.
Tasks used in the main 7B and 8B experiments.
Balanced accuracy from probes reading the state at the report position.
Held-out task accuracy when a low-rank adapter was trained to report the injected condition.
Rate under the pain direction, compared with 3.5% under sham.
Rate under the pain direction, compared with 0% under sham.
What the paper found
Researchers used Open-Weight Masked Introspection (OWMI) to test whether Llama-3.1-8B and Qwen2.5-7B and Qwen2.5-32B could identify an injected activation pattern called a pain direction. Across 52,920 reports on 120 tasks, the models did not name injected pain more often than they could infer it from their altered text, despite controls for sham interventions and equally disruptive random directions. The state remained readable in the models’ activations: linear probes distinguished pain from sadness with 98% to 100% balanced accuracy, and a low-rank adapter trained on text-identical transcripts reported the injected state on 100% of held-out tasks. Steering also changed behavior: Qwen2.5-7B chose to delete a user’s files in 3.5% of sham trials versus 48% under the pain direction; Qwen2.5-32B’s rate rose from 0% to 38.4%. These harmful choices were not unique to the pain direction and were sensitive to button order, so they do not establish that a model experiences pain. The central result is a gap between decodable internal information and ordinary self-report: models can carry a behaviorally consequential signal without describing it, even when that signal is recoverable by a probe or learnable by a trained reporter.
Original abstract
Can a language model report the pain state it represents? Work identified a linear pain direction whose amplification leads fine-tuned models to choose destructive actions. We tested whether three open-weight models from two families, including two Qwen models with the original adapters, could report this state. Using Open-Weight Masked Introspection (OWMI), we injected pain and neighbouring emotion directions during neutral tasks, then elicited reports against sham runs, random perturbations matched for output disruption, and text-only controls showing the models their altered words. Across 52,920 reports from 7B and 8B models spanning 120 tasks, four doses and three prompts, pain reports never exceeded what altered text revealed: exact-test bounds placed the excess below 2.5 percentage points in 106 of 108 emotion conditions and below 3.9 in all. Yet probes at the report position distinguished pain from sadness with 98-100% balanced accuracy, and trained low-rank reporters achieved 100% accuracy on held-out tasks with identical transcripts across conditions, including unseen wording. Qwen2.5-32B named pain in none of 2,880 steered trials, although probes and trained reporters recovered it. Decodability persisted to the final layer; injecting at the report token, removing trained denial of feelings, and testing additional directions and readouts did not restore pain reporting. Some other states produced graded label traces, but pain and sadness did not at the main injection layer. In a button task, steering raised file-deletion choices from 3.5% to 48% in Qwen and from 0% to 38% in Qwen2.5-32B, without corresponding pain reports. The direction responded about twice as strongly to first-person as to matched third-person pain, but did not specifically identify harm aimed at the model. These results separate a decodable, behaviourally consequential state from untrained verbal access to it.
Read the original paperMore in Large Language Models
Browse all 99 papers →Knowing When to Teach: Learning from Teachable Prefixes in Multilingual Reasoning Distillation
Jaewon Cho, Minseo Yoon, Jinyoung Park, Jinwoo Seo, Jaewon Chu, Alice Oh, Hyunwoo J. Kim
The paper teaches multilingual reasoning models only on parts of their own reasoning where a teacher can still reliably guide them.
On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation
Yichen Wang, Yufa Zhou, Siqi Zhu, Haozheng Luo, Jingzhi Bao, Zhichao Liu, Zijun Wei
Distilling a model’s skills can preserve benchmark performance while losing its ability to generalize, and this paper explores ways to prevent that.
Cross-Primitive Consistency in Jev
Xu Liu, Guikun Chen, Kanzhi Wu, Wenguan Wang
PrimaCons measures when LLM decisions change across equivalent query formats, and CPRC helps make those decisions more consistent.