LEANGUARD: Lean-Verified Guardrails for Tool-Using Agents
AuthorsSreehari Sankar, Debargha Ganguly, Vikash Singh, Mohsen Hariri, Weicong Chen, Yanyan Zhang, Shouren Wang, Wang Yang, Chaoda Song, Alexander Nemecek, Osama Zafar, Idan Davidovich, Yuting Shao, Kai Ye, Biyao Zhang, Xinpeng Li, Joe Stubbs, Erman Ayday, Jing Ma, Vipin Chaudhary
AffiliationsCase Western Reserve University
LeanGuard uses Lean-verified rules to block unsafe agent tool calls and sometimes repair them into safe alternatives.
Key results
At least this many policy-forbidden calls executed unguarded; LeanGuard dispatched none.
Reduced from 57.9% without the guard.
Reduced from 22.3% without the guard.
Lean proved repairs for 88 of 252 denials in offline replay.
Median time to find an admissible edit.
What the paper found
LeanGuard places a Lean 4 verifier between an agent and its tools, checking each proposed call against trusted backend evidence and execution history before dispatch. Its policies combine state predicates with the past-time operators PREVIOUS, ONCE, and SINCE; evaluation fails closed, and the decision procedure is proved equivalent to a LeanLTL semantics. When a call is denied, the agent can receive the violated rule and a corrective hint, or Lean metaprogramming can search for a nearby admissible edit and return a kernel-checked proof without consulting the model. Across five Qwen3.5 and Qwen3.8 models, unguarded agents executed at least 33 calls forbidden by the policy on τ²-bench, while LeanGuard dispatched none. On DTap, direct prompt-injection success fell from 57.9% to 5.9%, and indirect success from 22.3% to 3.8%. In offline replay, proof-guided repair found admissible edits for 88 of 252 argument-value denials, a 34.9% repair rate, with a median search time of 2 ms. Feedback increased recoveries, while overall τ²-bench task success with feedback was 70.9%, compared with 72.7% without a guard—a difference that was not statistically significant. The guarantee depends on accurate trusted inputs and reviewed policies, and tool-call checks cannot prevent harm carried only in an agent’s response text.
Original abstract
Tool-using language-model agents often execute actions that their domain policies forbid. We present LeanGuard, a runtime that checks every proposed tool call against a policy written in Lean4 before the call reaches the environment. Policies combine state predicates with scoped past-time temporal operators over the execution history, evaluation is fail-closed, and the decision procedure is proved equivalent to a LeanLTL semantics. A denied call can be corrected in two ways. With feedback, the agent receives the violated rule and a hint; with proof-guided repair, Lean metaprogramming searches for a nearby call that preserves the user's intent and proves that the policy admits it, without the model. On Tau-Bench with five Qwen models (0.8B-27B), unguarded agents execute at least 33 calls that the Lean policy forbids, while LeanGuard executes none; with feedback, task success does not change significantly. On DTap, LeanGuard reduces direct prompt-injection success from 57.9% to 5.9%, and in telecom, where benign tasks need no writes, benign success is unchanged. Feedback reduces repeated blocked calls and raises task success for four of five models over a bare denial. On replayed denials, repair proves an admissible edit for 88 of 252 denials caused by argument values, in a median of 2,ms.
Read the original paperMore in AI Safety
Browse all 54 papers →Is Forgetting Unified? Exploring Concept Unlearning in Unified Multimodal Models
Zhongkai Xue, Junkai Chen, Mingcong Lei, Rumo Zhang, Yixiao Liu, Zhimin Sun, Shu Wu, Qiang Liu, Lihua Zhang, Liang Wang
UniForget aims to make unified multimodal models both stop generating and stop recognizing selected concepts while preserving other capabilities.
LEASH: Stay Safe in Agentic Robotics
Qinfeng Zhu, Weiguang Zhao, Ke Sun, Xihao Guo, Yunxi Jiang, Rui Zhang, Lei Fan, Kaizhu Huang
RoboRisk tests how safely AI agents control robots, while LEASH adds checks intended to prevent dangerous plans and actions.
"Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents
Weiheng Wu, Junkai Chen, Zhensu Sun, Jieke Shi, Yunbo Lyu, David Lo
Caper tests what a coding agent’s command actually does in a sandbox before approving it, aiming to block harmful actions without stopping safe ones.