Liquid Gated Attention

Liquid Gated Attention, or LGA, is a solver-free temporal operator for irregularly sampled and long-horizon time series. It derives an input-dependent decay gate from liquid time-constant dynamics, approximates its integral with learnable endpoint interpolation, and stores history in a matrix-valued fast-weight associative memory. This lets LGA incorporate observed time intervals directly while retaining parallel computation: causal encoding uses a prefix scan, and non-causal encoding uses associative matrix multiplication, both with linear complexity in sequence length. A sequence-level normalization bounds cumulative decay and changes gradient attenuation from exponential to polynomial, improving long-horizon stability. The resulting LFormer backbone combines decoupled multi-head LGA with residual SwiGLU channel mixers and was evaluated across six tasks and sixteen datasets, including sequences of up to 17,984 steps, against models such as Transformer, Mamba, R-ODE, NCDE, and ContiFormer. LFormer achieved 71.3% average accuracy on six long-term classification datasets, reducing heart-rate regression MAE by 47.72% relative to the strongest baseline and reducing interpolation MSE on the irregular USH climate dataset by 83.33%. On a 4,096-step throughput test, LFormer reached 4652.90 sequences per second, while preserving linear memory scaling. Ablations show that normalization, input-aware gating, output gating, and head-specific interpolation are especially important for sparse, noisy, and extrapolative settings; however, fixing interpolation at 0.5, the classical trapezoidal rule, improved the Pendulum tracking result by 23.90% in MSE.