LLM2Cert: LLM Synthesis of Certificates with SMT/Lean Verification
AuthorsZhipeng Cao, Peiyang Song, Nikolaus Vertovec, Weiru Cui, Zhihao Yang, Mingkang Liu, Luxia Lin, Bai Xue, Alessandro Abate
AffiliationsInstitute of Software, Chinese Academy of Sciences (CAS) · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Language Technologies Institute, School of Computer Science, Carnegie Mellon University · Computing + Mathematical Sciences Department, California Institute of Technology · Department of Computer Science, University of Oxford · Gigaton Technologies Ltd · University of Chinese Academy of Sciences*Equal contribution. †Equal advising · Institute of Geographic Sciences and Natural Resources Research, CAS
LLM2Cert uses language models and formal tools to propose and verify mathematical safety and stability certificates for dynamical systems.
Key results
Benchmark size across stability, ROA, invariance, and safety tasks.
The framework verified 81 of 105 benchmark instances.
Percentage-point increase in verified coverage.
SMT proved 253 of 296 obligations on the fixed certificates.
Obligations repaired out of 26 failed first proof attempts.
What the paper found
LLM2Cert pairs language-model synthesis of Lyapunov and barrier certificates with a hybrid verification pipeline: Z3, cvc5, Yices, and dReal check suitable proof obligations, while unresolved or unsupported claims are sent to a Lean Proof-Repair agent that must produce a kernel-checked proof without changing the claim. Counterexamples and compiler diagnostics guide later certificate proposals, and the experiments use gpt-5.6-sol for the proposal and proof agents. The paper also introduces DYNACERTBENCH, a 105-instance benchmark spanning stability, region-of-attraction, invariance, and safety tasks, including high-dimensional and nonlinear systems with exact, unrelaxed verification conditions. LLM2Cert verifies 81/105 instances, or 77.1%, compared with 63/105 for one-shot proposal—a gain of 17.1 percentage points. On fixed certificates, SMT alone discharges 85.5% of obligations, while Lean addresses difficult cases the solvers leave open; structured repair completes 21 failed proof obligations, out of 26, with compiler-guided feedback. The central contribution is this separation of candidate discovery from sound verification: the language model proposes witnesses, but acceptance depends on definitive SMT results or machine-checked Lean proofs.
Original abstract
Lyapunov functions and barrier certificates give state-space guarantees of stability and safety, but synthesizing and verifying them---especially for nonlinear, high-dimensional systems---remains an open challenge. Existing pipelines verify candidates with numerical solvers (sum-of-squares programs), interval methods, or satisfiability-modulo-theories (SMT) solvers, which often require relaxations, are conservative, or scale poorly with model dimension and nonlinearity. These limitations have also shaped benchmark design: standard test suites are dominated by low-dimensional polynomial systems chosen to suit what solvers can handle rather than what engineering applications require. We address both sides of this challenge. First, we introduce DynaCertBench, a benchmark of 105 instances: 80 non-redundant systems curated from the literature, extended with high-dimensional and control-constrained variants and new nonlinearly coupled systems, comprehensively spanning stability, region-of-attraction, forward-invariance, and safety tasks. Second, we present LLM2Cert, an LLM-driven framework that synthesizes candidate certificates and verifies them against their proof obligations: each obligation is routed to an SMT solver or directly to the Lean Proof-Repair agent. On DynaCertBench, LLM2Cert verifies 81 of 105 instances ($77.1\%$), a $17.1$ percentage point improvement over one-shot proposal without feedback. On the 81 fixed certificates, LLM2Cert verifies 22 that All-SMT leaves unresolved. Code and data are available at https://github.com/FormalCert-AI/llm2cert.
Read the original paperMore in AI for Science
Browse all 52 papers →Can LLMs Uncover Hidden Physical Laws through Free Exploration?
Zhen Yang, Wen Wang, Xander Xu, Ying-Cong Chen
Researchers let LLMs freely explore simulated worlds with altered physics to see whether they could uncover the hidden laws.
Can Test-Time Memory Evolution Outperform Test-Time Training for Scientific Discovery?
Yepeng Liu, Shengmai Chen, Xuandong Zhao, Chengzhi Liu, Dawn Song, Xin Eric Wang, Yuheng Bu
The paper proposes improving AI-driven mathematical discovery by evolving its memory instead of retraining its weights at test time.
Automating ML for Science: Can Frontier Agents Climb Scientific Hills in the Wild?
Ming Zhong, Stacy Li, Nicholas Carlini, Matthew Jagielski
Agent4Sci tests how far frontier AI agents can improve real scientific ML models—and finds they need more than engineering tricks to make research-level progress.