NTH

LEANGUARD: Lean-Verified Guardrails for Tool-Using Agents

AuthorsSreehari Sankar, Debargha Ganguly, Vikash Singh, Mohsen Hariri, Weicong Chen, Yanyan Zhang, Shouren Wang, Wang Yang, Chaoda Song, Alexander Nemecek, Osama Zafar, Idan Davidovich, Yuting Shao, Kai Ye, Biyao Zhang, Xinpeng Li, Joe Stubbs, Erman Ayday, Jing Ma, Vipin Chaudhary

AffiliationsCase Western Reserve University

November 7, 2026 2 min read
Watch on YouTube
The one-line take

LeanGuard uses Lean-verified rules to block unsafe agent tool calls and sometimes repair them into safe alternatives.

Key results

33
Forbidden τ²-bench calls executed without guard

At least this many policy-forbidden calls executed unguarded; LeanGuard dispatched none.

5.9%
DTap direct attack success with LeanGuard

Reduced from 57.9% without the guard.

3.8%
DTap indirect attack success with LeanGuard

Reduced from 22.3% without the guard.

34.9%
Argument-value denial repair rate

Lean proved repairs for 88 of 252 denials in offline replay.

2 ms
Median proof-guided repair search time

Median time to find an admissible edit.

What the paper found

LeanGuard places a Lean 4 verifier between an agent and its tools, checking each proposed call against trusted backend evidence and execution history before dispatch. Its policies combine state predicates with the past-time operators PREVIOUS, ONCE, and SINCE; evaluation fails closed, and the decision procedure is proved equivalent to a LeanLTL semantics. When a call is denied, the agent can receive the violated rule and a corrective hint, or Lean metaprogramming can search for a nearby admissible edit and return a kernel-checked proof without consulting the model. Across five Qwen3.5 and Qwen3.8 models, unguarded agents executed at least 33 calls forbidden by the policy on τ²-bench, while LeanGuard dispatched none. On DTap, direct prompt-injection success fell from 57.9% to 5.9%, and indirect success from 22.3% to 3.8%. In offline replay, proof-guided repair found admissible edits for 88 of 252 argument-value denials, a 34.9% repair rate, with a median search time of 2 ms. Feedback increased recoveries, while overall τ²-bench task success with feedback was 70.9%, compared with 72.7% without a guard—a difference that was not statistically significant. The guarantee depends on accurate trusted inputs and reviewed policies, and tool-call checks cannot prevent harm carried only in an agent’s response text.

Original abstract

Tool-using language-model agents often execute actions that their domain policies forbid. We present LeanGuard, a runtime that checks every proposed tool call against a policy written in Lean4 before the call reaches the environment. Policies combine state predicates with scoped past-time temporal operators over the execution history, evaluation is fail-closed, and the decision procedure is proved equivalent to a LeanLTL semantics. A denied call can be corrected in two ways. With feedback, the agent receives the violated rule and a hint; with proof-guided repair, Lean metaprogramming searches for a nearby call that preserves the user's intent and proves that the policy admits it, without the model. On Tau-Bench with five Qwen models (0.8B-27B), unguarded agents execute at least 33 calls that the Lean policy forbids, while LeanGuard executes none; with feedback, task success does not change significantly. On DTap, LeanGuard reduces direct prompt-injection success from 57.9% to 5.9%, and in telecom, where benign tasks need no writes, benign success is unchanged. Feedback reduces repeated blocked calls and raises task success for four of five models over a bare denial. On replayed denials, repair proves an admissible edit for 88 of 252 denials caused by argument values, in a median of 2,ms.

Read the original paper

More in AI Safety

Browse all 54 papers →
02Safety

LEASH: Stay Safe in Agentic Robotics

Qinfeng Zhu, Weiguang Zhao, Ke Sun, Xihao Guo, Yunxi Jiang, Rui Zhang, Lei Fan, Kaizhu Huang

RoboRisk tests how safely AI agents control robots, while LEASH adds checks intended to prevent dangerous plans and actions.

Read analysis