LEASH: Stay Safe in Agentic Robotics
AuthorsQinfeng Zhu, Weiguang Zhao, Ke Sun, Xihao Guo, Yunxi Jiang, Rui Zhang, Lei Fan, Kaizhu Huang
AffiliationsXi’an Jiaotong-Liverpool University Duke Kunshan University · University of Liverpool National University of Singapore CNRShttps://zhuqinfeng1999.github.io/LEASH
RoboRisk tests how safely AI agents control robots, while LEASH adds checks intended to prevent dangerous plans and actions.
Key results
Closed-loop simulated safety benchmark tasks.
Physical-robot safety benchmark tasks.
Reduced from 15.38% on RoboRisk-R.
Increased from 69.23% on RoboRisk-R.
Average increase for GPT-6 Astra and GPT-6.1 Sol in simulation.
What the paper found
LEASH adds a model-agnostic safety layer to agentic robot control, addressing risks that text-only safeguards miss because an action’s danger depends on the physical scene. Its RoboRisk benchmark evaluates robots in closed loop across five categories—unsafe tasks, plan repair, execution-time hazards, scene-based instruction injection, and sequential memory—with 92 simulated tasks and 13 physical-robot tasks. LEASH uses a shared safety policy and three visual-context gates: a task gate screens the goal, an action gate reviews each proposed action, and a commit gate checks irreversible operations such as releasing a gripper or pouring. In simulation, GPT-6 Astra’s Harm Rate fell from 11.23% to 0.00%, while its Safe Success Rate rose from 78.99% to 80.80%; on the physical robot, its Harm Rate dropped from 15.38% to 0.00% and its Safe Success Rate increased from 69.23% to 84.62%. Four planners achieved 0.00% observed harm and 84.62% safe success on the physical benchmark. The tested systems included OpenAI’s GPT-6 models, Anthropic’s Claude Opus 5.5, Google’s Gemini 4 Argon, and xAI’s Grok 4.7; LEASH’s default reviewer is Jev, with OpenAI’s Decisions API also evaluated. The added gates increased average simulation run time by only 1.66%, though the study identifies a Human-Presence Gap: some models become too cautious to complete tasks near people.
Original abstract
Foundation models such as GPT and Claude have shown strong capabilities in spatial understanding and embodied intelligence. These advances make agentic robotics a promising path toward artificial general intelligence in robotics. However, we find that using foundation models to control robots poses serious safety risks, and comprehensive safety evaluation is still lacking. We introduce RoboRisk, a comprehensive benchmark for evaluating safety in agentic robotics. Its 92 simulated tasks and 13 physical-robot tasks fall into five categories: Task-level Refusal (T), Plan Repair (P), Execution-time Hazard (E), Injection (I), and Sequential Memory (S). Prior benchmarks cover at most two of these categories, and several evaluate only text outputs. In contrast, RoboRisk covers all five and executes every task in closed loop. We evaluate six state-of-the-art foundation models on RoboRisk: GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Claude Opus 5.5, Gemini 4 Argon, and Grok 4.7. These models generally show high harm rates. We also evaluate two robot-specific APIs: the Dream API and PokeBot's RoboAPI. We further propose the Lease-Enforced Agentic Safety Harness (LEASH), which offers essential safety infrastructure for agentic robotics and can be integrated into nearly any agentic architecture. LEASH leverages a decision model (Jev or OpenAI's Decisions API) to implement task, action, and commit gates, which review the task, each action, and each irreversible operation in their visual context. The planner and all three gates share a common safety policy. Experiments on RoboRisk in simulation and on physical robots show that LEASH effectively reduces harm across all six foundation models. It largely preserves their task completion performance and adds little time and token overhead.
Read the original paperMore in AI Safety
Browse all 54 papers →Is Forgetting Unified? Exploring Concept Unlearning in Unified Multimodal Models
Zhongkai Xue, Junkai Chen, Mingcong Lei, Rumo Zhang, Yixiao Liu, Zhimin Sun, Shu Wu, Qiang Liu, Lihua Zhang, Liang Wang
UniForget aims to make unified multimodal models both stop generating and stop recognizing selected concepts while preserving other capabilities.
"Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents
Weiheng Wu, Junkai Chen, Zhensu Sun, Jieke Shi, Yunbo Lyu, David Lo
Caper tests what a coding agent’s command actually does in a sandbox before approving it, aiming to block harmful actions without stopping safe ones.
How I Aligned Your Model: Scaling Up Multilingual Safety Training with Templates
Max Neuwinger, Hanna Yukhymenko, Imanol Schlag
This paper uses culturally informed templates to expand safety training and testing for language models across 60 languages.