Specula: Scaling formal specifications for autonomous model checking of system code

Specula, developed by researchers from Nanjing University, the University of Illinois Urbana-Champaign, Microsoft Research Asia, and the University of British Columbia, is a push-button agentic system for model checking real-world concurrent and distributed software. Using coding agents such as Claude Code, Codex, and Copilot CLI—defaulting to Claude Opus-4.8—it generates TLA+ invariants and implementation-level models, then combines TLC model checking with automated code instrumentation, trace validation, bidirectional repair, and self-evolving loops. This design addresses hallucinated specifications and reward hacking by requiring models to admit code traces while rejecting states that violate invariants. Across 48 open-source systems, including MongoDB, ScyllaDB, GCC libgomp, and SONiC, Specula found 249 bugs; 200 emerged through model checking, with breadth-first search finding 187 and a median counterexample length of 9 steps. On SysMoBench, Specula with Opus-4.8 achieved 100% on syntax, runtime correctness, model-code conformance, invariant satisfaction, and overall quality, while its controlled comparison found 62 bugs versus 2 for raw Claude Code and 3 for Claude Code equipped only with TLA+ tools. The system reproduced 98% of violations judged to be real bugs, required a median of 3.69 hours per system, and had a median token cost of $57. The results show that autonomous formal reasoning can scale beyond handcrafted specifications, although it remains dependent on strong LLMs and does not provide complete guarantees against modeling omissions.