Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding

Domino, from EPIC Lab at Shanghai Jiao Tong University and Huawei collaborators, tackles a core bottleneck in speculative decoding for large language models such as Qwen3-4B and Qwen3-8B: autoregressive drafters like EAGLE-3 preserve causal token dependencies but pay a sequential cost, while fully parallel drafters like DFlash cut latency but weaken intra-block dependency modeling. Domino decouples these two goals by using a parallel draft backbone to generate block-level preliminary distributions in one pass, then adding a lightweight causal correction branch with a GRU-based encoder and a low-rank logit-space head to inject prefix information without re-running a full LM head. The training recipe is equally novel: a teacher-forced causal encoder avoids noisy self-generated prefixes, and a base-anchored curriculum first optimizes the parallel backbone before gradually shifting weight to the corrected final logits. On math, code, and chat benchmarks including GSM8K, MATH-500, AIME25, HumanEval, MBPP, LiveCodeBench, MT-Bench, and Alpaca, Domino improves average acceptance length and speedup over representative baselines; on Qwen3-8B it reaches 5.49× average end-to-end speedup under the Transformers backend and up to 5.8× throughput speedup in SGLang, with a standout 7.92× speedup on GSM8K versus 5.21× for DFlash. Ablations show the Domino head alone raises average acceptance length from 3.49 to 4.19 and speedup from 2.84× to 3.31×, confirming that lightweight causal correction is the key mechanism.