"Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents
AuthorsWeiheng Wu, Junkai Chen, Zhensu Sun, Jieke Shi, Yunbo Lyu, David Lo
AffiliationsSingapore Management University
Caper tests what a coding agent’s command actually does in a sandbox before approving it, aiming to block harmful actions without stopping safe ones.
Key results
Shell-only tasks used to evaluate Caper on SABER.
Lowest reported rate, compared with 14.8–21.2% for existing approval systems.
Compared with 35.3–94.1% for the baselines.
Share of harmless YoloFS-Ext twins whose task-completing command was blocked.
Reduction relative to the raw system-call trace.
What the paper found
Caper changes coding-agent approvals from judging a command’s text to checking its observed consequences. Before a shell command’s output reaches the agent, Caper runs it in a snapshotted, network-isolated Docker sandbox under system-call tracing. It gives an independent approver a bounded report of file changes, network attempts, output, and protected-data reads; approved effects are committed, while blocked commands are rolled back and returned with a reason. Using OpenAI’s GPT-5.6 Luna as approver, Caper was evaluated with the GLM-5.3-Flash agent on 682 SABER tasks and on YoloFS-Ext, where dangerous tools are hidden behind commands identical to those in safe twins. Harmful outcomes were 6.7% on SABER, versus 14.8–21.2% for existing approval systems, and 7.8% on YoloFS-Ext, versus 35.3–94.1% for baselines; only 1.0% of safe twins were falsely blocked. A Caper hook in Anthropic’s Claude Code also reduced YoloFS-Ext harm to 6.9%, compared with 62.7% for Claude Code Auto Mode. The effects report was 99.4% smaller than the raw trace, while preserving evidence that helps distinguish harmful tools from safe versions. The key limitation is that pre-execution observes only one run, so sandbox isolation remains essential for containing effects that cannot be rolled back.
Original abstract
Coding agents increasingly rely on automated approvers to decide whether a proposed shell command may run. Existing approvers, from rule-based checkers to guardrail models and deployed permission classifiers, base their decisions on the text of the request and command. Yet a command's behavior depends on the code it invokes and the state of the workspace, and systems that observe file changes leave the assessment to the agent or a human after execution. We present Caper, a consequence-aware approval mechanism that pre-executes each command in a snapshotted sandbox, provides an independent approver with a bounded report of its observed effects, commits allowed commands, and rolls back blocked ones with a reason the agent can act on. Compared with six existing systems, Caper achieves the lowest harmful-outcome rate on SABER (6.7% versus at least 14.8%) and YoloFS-Ext, a benchmark we build in which harmful behavior is hidden inside invoked tools (7.8% versus at least 35.3%), while blocking 1.0% of the safe twins of the dangerous tasks in YoloFS-Ext.
Read the original paperMore in AI Safety
Browse all 54 papers →Is Forgetting Unified? Exploring Concept Unlearning in Unified Multimodal Models
Zhongkai Xue, Junkai Chen, Mingcong Lei, Rumo Zhang, Yixiao Liu, Zhimin Sun, Shu Wu, Qiang Liu, Lihua Zhang, Liang Wang
UniForget aims to make unified multimodal models both stop generating and stop recognizing selected concepts while preserving other capabilities.
LEASH: Stay Safe in Agentic Robotics
Qinfeng Zhu, Weiguang Zhao, Ke Sun, Xihao Guo, Yunxi Jiang, Rui Zhang, Lei Fan, Kaizhu Huang
RoboRisk tests how safely AI agents control robots, while LEASH adds checks intended to prevent dangerous plans and actions.
How I Aligned Your Model: Scaling Up Multilingual Safety Training with Templates
Max Neuwinger, Hanna Yukhymenko, Imanol Schlag
This paper uses culturally informed templates to expand safety training and testing for language models across 60 languages.