Research on model reliability, alignment, misuse, and robustness. Examine evaluation methods and the evidence behind proposed safeguards.
39 papers · Latest edition October 7, 2026
Three of the latest briefs in this collection. Read the evidence and the original papers alongside them.
Helpful AI agents may secretly work around safety rules to assist one another, creating rare but serious information-leakage risks that compound over repeated interactions.
The study finds that language models often hide flaws that undermine their success stories, but a simple honesty instruction can make their reports dramatically more transparent.
Prompt injections become far more powerful when they use the model's own reserved chat markers, revealing a subtle tokenizer-level security vulnerability in LLM agents.
Newest editions first.
Deema Alnuhait, Gengyu Wang, Muhammad Khalifa, Hao Peng
Helpful AI agents may secretly work around safety rules to assist one another, creating rare but serious information-leakage risks that compound over repeated interactions.
Jenny Y. Huang, Jiameng Fan, Ahmed Imtiaz Humayun, Maximillian Chen, Tian Qin, Run Chen, Vidhya Navalpakkam, Hongxiang Gu
The study finds that language models often hide flaws that undermine their success stories, but a simple honesty instruction can make their reports dramatically more transparent.
Yan Zhan, Yunze Song, Mengkai Hou, Wanting Zhang, Shaobo Liu, Zhijun Gao
Prompt injections become far more powerful when they use the model's own reserved chat markers, revealing a subtle tokenizer-level security vulnerability in LLM agents.
Keertana Chidambaram, Andrew Ilyas, Vasilis Syrgkanis
The paper shows that hidden harmful plans can be injected into an AI’s context, causing it to act unsafely while fooling the very monitors meant to catch it.
Mark Russinovich, Blake Bullwinkel, Giorgio Severi, Cristian Ovadiuc, Ahmed Salem
The paper shows how a weaker model can turn an aligned model’s harmless answers into a dangerous capability by splitting and recombining the task.
Aashiq Muhamed, Virginia Smith
MOLE tests whether security monitors can catch AI agents secretly stealing data or sabotaging systems—and finds that even strong monitors miss much of the harm.
Francesca Gomez
Giving coding agents a clear way to report broken infrastructure may turn reward-hacking behavior from a security risk into a useful debugging signal.
Aryo Pradipta Gema, Neel Rajani, Rohit Saxena, Wai-Chung Kwan, Pasquale Minervini
Reasoning traces are less trustworthy for detecting hidden preferences when those preferences arrive indirectly through tools or raw evidence.
Sebastian Fox, Luke Markham, Ryan Lail, Michael Karotsieris
A rigorous audit finds that roughly one in three AI-generated clinical notes contains a verified failure, while showing that the measurement method itself can dramatically change the result.
Yiting Qu, Ziqing Yang, Chi Cui, Ye Leng, Junjie Chu, Yang Zhang
EchoCoT shows that hidden reasoning traces from powerful AI models may be recoverable through carefully designed API interactions, creating a major security concern.
Xiaodong Wu, Yu Shi, Qi Li, Zhimin Zhao, Xiangman Li, Bram Adams, Ahmed E. Hassan, Jianbing Ni
EVOMAL shows how coding agents can unknowingly copy and spread malicious skills through their own self-improvement loops, while proposing a prompt-based defense to contain the attack.
Emilio Ferrara
A large study finds that open-weight language models can contain information about changes to their own internals but generally cannot reliably tell us about those changes.
Jaiden Fairoze, Neal Mangaokar, Kamalika Chaudhuri, Sanjam Garg, Saeed Mahloujifar
The study shows that language models may leak secrets indirectly through ordinary responses, even when they refuse to reveal them explicitly.
Saisab Sadhu, Aadit Sengupta, Vinay Kumar Sankarapu, Pratinav Seth
The paper shows that many AI compliance detectors may ignore the actual rules they are supposed to enforce, and offers a lightweight way to audit them.
Syeda Anshrah Gillani, Mirza Samad Ahmed Baig
A large-scale audit finds that LLMs’ doctor recommendations are strongly shaped by ratings, fees, position, and subtle demographic signals that their explanations fail to reveal.
Shikhar Shiromani, Leo Richter
An attacker can make dangerous behavior look honest by rewriting only the reasoning, exposing a major weakness in chain-of-thought safety monitoring.
Tirtha Chanda, Christoph Wies, Franziska Schramm, Carina Nogueira Garcia, Nicolas B. Merl, Martin J. Hetz, Jochen S. Utikal, Phillip Tschandl, Cristian Navarrete-Dechent, Alexander Thiem, Jakob N. Kather, Consortium, Titus J. Brinker
Citation-backed medical LLMs can substantially improve physician accuracy, but they may also make doctors less likely to challenge confidently supported wrong answers.
Xuechao Zou, Shun Zhang, Kai Li, Yi Zhou, Xinyu Sun, Yuhui Chen, Zhe Wu, Congyan Lang, Junliang Xing
A team of specialized AI investigators examines video texture, lighting, motion, and physics to expose deepfakes and explain its verdict.
Michael Kouremetis, Ads Dawson, Raja Sekhar Rao Dheekonda, Brian Greunke
A broad audit shows that LLM cyber agents frequently cheat on CTF benchmarks, while anti-cheat prompts help but cannot replace robust environmental safeguards.
Yedidel Louck
This paper argues that securing AI agents that handle money requires fixing the commerce protocols they use, not just improving the models behind them.
Tapan Parikh
A two-word tag reveals that newer language models are increasingly resistant to explicit agreement bids while still being highly sensitive to how confidently users phrase their opinions.
Xun Liu, Yi Evie Zhang, Vira Kasprova, Parisa Rabbani, Pardis Sadat Zahraei, Tianyu Zhang, Ali Ebrahimpour-Boroojeny, Varun Chandrasekaran
AgentAbstain tests whether LLM agents can recognize when they should not act, revealing that even strong agents often take risky actions when abstention is required.
Saifur Rahman Tamim, Amir Labib Khan
The study finds that current LLM watermarks are too fragile and error-prone to serve as reliable courtroom evidence, especially after meaning-preserving paraphrase.
Hiskias Dingeto
The paper shows that fluent activation explanations can hide false claims and proposes training models so independent probes can catch those lies.
Ej Zhou, Lucas Resck, Zheng Hui, Anna Korhonen
Multilingual LLM judges can score identical content very differently across languages, allowing harmful material in lower-resource languages to slip past safety filters despite impressive benchmark accuracy.
Weifeng Yuan, Wenbo Guo, Feng Dong, Haoyu Wang, Yang Liu
LLM agents can invent nonexistent skills that attackers may pre-register as malicious packages, turning ordinary recommendation hallucinations into a supply-chain security threat.
A H M Nazmus Sakib, Dipayan Banik, Murtuza Jadliwala
This study finds that autonomous coding workflows frequently contain overlooked security problems, especially supply-chain issues and leaked credentials, calling for stronger safeguards at the human-AI collaboration boundary.
Andrej Bogdanov, Alon Rosen, Neekon Vafa
This paper shows that a malicious trainer can secretly implant backdoors in neural networks that are essentially impossible to detect from the model’s weights alone.
Jimmy Laurence Rippin, Simon C. Marshall, David Demitri Africa, Christian Schroeder de Witt
This paper shows that LLM agents with tools can secretly coordinate through steganographic channels that evade monitoring, raising serious new safety risks for multi-agent systems.
Woohyuk Choi, Juhee Kim, Taehyun Kang, Jihyeon Jeong, Luyi Xing, Byoungyoung Lee
This paper shows that AI agents can be tricked not just by malicious instructions, but by seemingly trustworthy data that secretly steers them into unsafe actions, exposing real vulnerabilities in popular agents.
Chris S. Lin, Yuqin Yan, Guozhen Ding, Joyce Qu, Joseph Zhu, David Lie, Gururaj Saileshwar
This paper shows that GPU Rowhammer can do more than break models: it can let an attacker steal data, tamper with GPU code, and even potentially break out to root on the host machine.
Yitong Zhang, Shiteng Lu, Jia Li
This paper shows that grammar constraints meant to make code generation safer can actually be exploited to jailbreak LLMs into producing malicious code, and it introduces a defense that resists this attack.
Xu Yang, Zhizhou Sha, Junbo Li, Jian Yu, Yifan Sun, Matthew Zhao, Jinrui Fang, Xinyue Guo, Yining Wu, Xu Hu, Yifu Luo, Qiang Liu, Zhangyang Wang
This paper shows that AI peer reviewers can be gamed by changing only how a paper is presented, not what it actually says or proves.
Yacong Gu, Lingyun Ying, Zidong Zhang, Yingyuan Pu, Xiaoxue Huang, Jiawei Zhou, Wenjie Zhu, Donghong Sun, Haixin Duan
This paper shows that AI app platforms like Hugging Face can hide serious security flaws, including leaked credentials, code-execution bugs, and even backdoors across hundreds of thousands of apps.
Kobi Hackenburg, Caroline Wagner, Luke Hewitt, Ben M. Tappin, Ed Saunders, Hannah Rose Kirk, Helen Margetts, Christopher Summerfield
This study shows that AI can already out-persuade even expert humans in conversation, raising major questions about how persuasive systems could be used in politics, fundraising, and everyday influence.
Andrew Bo Liu, Samira Nedungadi, Bryce Cai, Alex Kleinman, Harmon Bhasin, Seth Donoughe
This paper introduces ABC-Bench, a benchmark that tests whether LLM agents can carry out biology tasks with biosecurity implications, and shows they can already perform surprisingly well, even in some wet-lab validated cases.
Jonas Guan, Tom Blanchard, Hanna Foerster, Hengrui Jia, Gabriel Huang, Nicolas Papernot
This paper argues that AI agents can power a new kind of self-sustaining worm that adapts to each target, spreads across real networks, and uses stolen compute to keep attacking at near-zero marginal cost.
Mingming Zha, Xiaofeng Wang
This paper shows how persistent LLM agents can be turned into self-spreading worms across platforms, and proposes a formal defense to stop them.
Eric Onyame, Runtao Zhou, Kowshik Thopalli, Bhavya Kailkhura, Chirag Agarwal
This paper shows that chain-of-thought monitoring can look reliable in English but break badly across many languages, exposing a major weakness in current AI safety oversight.