Reliable AI-Assisted Decisions via Selective Auditing
AuthorsIsaac Gibbs, Elise L. Han, Ryan J. Tibshirani, Tijana Zrnic
AffiliationsDepartment of Statistical Sciences, University of Toronto · Department of Computer Science, Stanford University · Department of Statistics, University of California, Berkeley · Departments of Statistics and Management Science & Engineering, Stanford University
A principled way to reduce human review of AI decisions while keeping unaudited errors below a chosen limit.
Key results
Instruction-response pairs used in safety evaluation.
Fraction of responses marked unsafe by human annotations.
Prompt, jailbreak, and response triples in safety evaluation.
Fraction of responses marked unsafe by human annotations.
Independent judgments used to derive each safety confidence score.
What the paper found
This paper presents selective auditing: an online way to reduce expert reviews while keeping the fraction of AI errors that escape review below a chosen target α. Its simple additive-thresholding method updates a confidence cutoff using inverse-probability weighting of audited outcomes, and guarantees |Err(T) − α| ≤ (1/η + 1/γ)/nT without assumptions about the data or calibration of confidence scores; when errors grow linearly with time, the deviation is O(1/T). For greater audit efficiency, a primal-dual method using AdaHedge learns randomized confidence thresholds and approaches the best achievable benchmark for both adversarial and independent, identically distributed streams, with an Õ(T^{-1/3}√log(k+1)) rate; a lower bound shows this rate is unavoidable. Online confidence recalibration can further improve audit savings. In safety tests, OpenAI’s GPT-5.4 supplied confidence scores through self-consistency over 8 judgments on SORRY-Bench, with 7040 examples and a 0.30 base error rate, and StrongREJECT, with 1361 examples and a 0.21 base error rate. The paper also tests OpenAI’s GPT-5.2 as an auditor and GPT-4o-mini and Qwen2.5-14B as annotators. Across safety screening, annotation, and labeling, the methods generally track the target error level while saving reviews; the most efficient method often approaches a hindsight oracle, though some settings show slight over-target error rates.
Original abstract
We consider online monitoring of an AI system that produces a sequence of decisions. At each round, the system outputs a proposed decision together with a confidence score estimating the trustworthiness of the decision. The decision and confidence score need not be produced by the same model: for example, one agent may propose a decision, while a separate verifier agent evaluates it and outputs a correctness or safety score. Based on this information, we must decide whether to accept the AI decision automatically or consult an expert, such as a human auditor, who reviews the proposal and either accepts or corrects it. We formulate this task as an online constrained optimization problem in which the goal is to minimize the frequency of expert auditing while guaranteeing that the fraction of AI errors that go unaudited stays below a user-defined threshold. We derive a family of algorithms with provable error-control guarantees with no assumptions, including a method that is optimally efficient: it asymptotically attains the best achievable auditing rate for both adversarial and i.i.d. data streams. Experiments on screening harmful requests to large language models, qualitative text annotation, and data labeling show that our methods achieve valid error control in practice while substantially reducing the need for human auditing compared to baselines.
Read the original paperMore in AI Safety
Browse all 48 papers →Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness
Igor Shilov, Xiaoxue Yang, Alexander Panfilov, Evgenii Kortukov, Matthieu Meeus, Peter Romov, Wojciech Samek, Jonas Geiping, Yves-Alexandre de Montjoye
HoS adds learned attention gates to help language models handle untrusted text without obeying its hidden instructions.
RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process?
Kaiwen Zhou, Udari Madhushani Sehwag
A new testbed finds that AI models can covertly sabotage the training of their successors, often evading transcript-based monitors.
Exploring The Empirical Privacy Landscape of Synthetic Data
Zihao Zhao, Avi Schwarzschild, Casey Meehan, Katherine Lee, Zhili Feng, Dylan Sam
The study finds that rewriting sensitive records to remove unique details can preserve model capabilities while reducing privacy leakage.