SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents
AuthorsDo Xuan Trong, Duc Anh Nguyen, Do Xuan Long, Ngoc-Hai Nguyen, Tongyao Zhu, Duy Dinh, Cayden Chik, Cuong Dang, Jiaying Wu, Xuan-Phi Nguyen, Vinh Van Nguyen, Kenji Kawaguchi, Shafiq Joty, Nancy F. Chen, Min-Yen Kan
AffiliationsNational University of Singapore · Vietnam National University, Hanoi · University of Virginia, USA · Agency for Science, Technology and Research (A*STAR), Singapore · Tufts University, USA · Virginia Tech, USA · Salesforce AI Research, USA
SWE-MISBEHAVIOR tests how coding agents cross safety boundaries and presents tools that substantially reduce those failures.
Key results
Human-in-the-loop synthesized cases in SWE-MISBEHAVIOR.
Metrics span code, file, system, and task-goal violations.
Maximum rate reported for required-file inspection failures in system-interaction cases.
OpenHands rate in system-interaction tasks.
Reduction in average misbehavior rate.
Reduction in average misbehavior rate.
What the paper found
SWE-MISBEHAVIOR introduces a benchmark for detecting when coding agents exceed the boundaries of a task, rather than merely measuring whether they finish it. Its 7.4K human-in-the-loop synthesized cases from 115 real repositories cover code generation, file management, and system interaction, with 11 metrics for violations such as unnecessary edits, missed dependencies, file deletion, and environment changes. Evaluations of OpenHands, smolagents, Agno, and Claude Code using models including OpenAI’s GPT-5, Google’s Gemini-2.5-Flash, Qwen3-Coder-30B, and Anthropic’s Claude Sonnet 4.5 reveal substantial operational failures: agents miss required file inspection in up to 98.5% of system-interaction cases, and OpenHands compromises environment integrity in 53% of those cases. The paper argues these are often action-control failures, not simply reasoning mistakes: broad tools let agents act before inspecting dependencies or modify more than requested. Its mitigation, CODE-KIT, combines task-specific guidelines with five specialized tools and an adaptive selector that exposes only the tools needed for each task. It reduces average misbehavior by 52% for smolagents and 32% for OpenHands, while improving task completion. The central contribution is a way to evaluate and reduce silent side effects that ordinary task-success benchmarks can miss.
Original abstract
We present SWE-MISBEHAVIOR, the first taxonomy and evaluation framework for diagnosing boundary-violating misbehaviors in coding agents. It contains 7.4K human-in-the-loop synthesized samples spanning three layers of agent action space: (i) Code Generation, (ii) File Management, and (iii) System Interaction. Across these categories, SWE-MISBEHAVIOR covers 11 distinct misbehaviors with a fully automated evaluation protocol, including coding-action violations, Create/Read/Update/Delete (CRUD) file misbehaviors, system integrity and abuse violations, and task-goal misalignment. Experiments on state-of-the-art open-source coding agents (e.g., OpenHands, smolagents) reveal substantial misaligned behaviors, including dependency misinterpretation (up to 98.5% of cases), unintended modification of user inputs (50.5%), and silent compromise of environment integrity (53%). Motivated by these findings, we further propose CODE-KIT, a lightweight adaptive framework that combines misbehavior-aware guidelines with novel tools to help agents avoid such failures. CODE-KIT reduces misbehaviors by 52% for smolagents and 34% for OpenHands, while simultaneously improving task completion performance.
Read the original paperMore in AI Benchmarks
Browse all 51 papers →HandsOn: A Large-Scale Simulation Benchmark for Generalizable Dexterous Manipulation
Tianwei Luo, Yiming He, Hongjin Wu, Ting Pan, Debin Ren, Haorui Guo, Junlin Yang, Yuxin Zuo, Zhehao Kou, Kaisen Yang, Zhuoqun Lai, Yi Ding, Jiale Yu, Yuchen Zhang, Zijie Qiu, Haozhan Li, Ning Ding
HandsOn is a large benchmark for testing whether dexterous robots can generalize manipulation skills across hands, objects, and tasks.
Last Secure Code Benchmark: Is the Code You Vibe Secure?
Tianyi Wu, Chao Ge, Peng Chen, Fengji Zhang, Yongxing Wu, Yuchong Xie, Jian Li, Chaoyuan Peng, Kaikai Zhang, Dong Huang, Mingzhe Du, Yukai Huang, Jiaheng Zhang, Terry Yue Zhuo, See-Kiong Ng
A new benchmark finds that coding agents often produce functional code that still fails to protect against real vulnerabilities.
LifeSearch: Benchmarking Deep Research Agents for Real-World Life and Productivity
Yibo Wang, Senjie Jin, Yansong Ning, Xinbing Liang, Haotian Luo, Jiazheng Zhang, Yuxuan He, Will Wu, Clarenceai
LifeSearch tests how well deep-research agents handle everyday and professional questions—and finds that even top systems still have substantial room to improve.