Last Secure Code Benchmark: Is the Code You Vibe Secure?

The Last Secure Code Benchmark, or LSCBench, tests whether coding agents produce code that is both functional and secure when users specify behavior but say nothing about security. It includes 450 tasks built from 150 real vulnerabilities, across seven languages and 24 CWE categories, with each vulnerability tested at function, file, and repository scope using hidden functional and attack-based security tests. Across six agents—including Anthropic’s Claude Fable 5, OpenAI’s GPT-5.6-Sol, Alibaba’s Qwen3.8-Max, DeepSeek-V4-Pro, GLM-5.3, and Kimi-K3—functional pass rates are generally high, but Claude Fable 5, the strongest on joint success, passes both test suites on only 25.1% of tasks and secures all three scopes for just 12% of scenarios. The analysis finds that agents more often include local safeguards, such as bounds checks, than infer unstated deployment policies governing access, data exposure, or permitted destinations; more reasoning turns alone do not reliably close that gap. A one-sentence security reminder substantially improves joint success for several agents, but does not eliminate the vulnerability problem. LSCBench’s central contribution is to measure secure code generation under realistic, security-neutral “vibe coding” requests rather than treating functional correctness as evidence of safe software.