From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models

From Scores to Gibbs Correctors introduces Gibbs-Accelerated Discrete Diffusion, or GADD, a predictor-corrector sampler for uniform-rate discrete diffusion models that converts the concrete score function directly into Gibbs posterior conditionals, eliminating any extra training beyond standard score estimation. The central novelty is theoretical: GADD is proved to achieve overall sampling complexity O(polylog(ε−1)), the first diffusion-based sampler for uniform-rate discrete models with logarithmic dependence on target TV accuracy ε, whereas prior deterministic samplers such as Euler, Tweedie τ-leaping, and higher-order Runge-Kutta variants remain polynomial in ε−1. The method alternates an Euler predictor with a random-scan Gibbs corrector whose posterior is computed from the score estimator via closed-form normalization, so one forward score pass yields all token-wise conditional probabilities. The analysis replaces Girsanov-based arguments with an induction framework that tracks error propagation across predictor steps and explicitly handles inaccurate correctors, then shows how the diffusion warm start reduces both Gibbs mixing error and score-estimation error. On synthetic spiky distributions, WikiText103 with a SEDD Uniform model, and zero-shot conditional music completion on Lakh pianoroll data, GADD consistently improves sample quality and wall-clock efficiency over vanilla Euler and CTMC correctors; for example, in text sampling at NFE 256 it lowers generative perplexity from 275.4 for vanilla Euler to 149.6, while also beating the θ-Trapezoidal sampler. The paper also proves that CTMC correctors still incur O(poly(ε−1)) complexity, highlighting why Gibbs-based correction is asymptotically superior in this setting.