Neuronal Stochastic Attention Circuit (NSAC) for Probabilistic Representation Learning

The paper introduces the Neuronal Stochastic Attention Circuit, or NSAC, a continuous-time attention layer that replaces deterministic attention logits with the exact solution of an Ornstein–Uhlenbeck stochastic differential equation, using input-dependent gates borrowed from C. elegans Neuronal Circuit Policies. This design turns attention logits into a Gaussian process over time and pushes that uncertainty through a logistic-normal attention distribution, so the model produces probabilistic outputs natively rather than via post-hoc calibration. Training combines Gaussian negative log-likelihood with an epistemic-separation regularizer that increases variance on synthetic out-of-distribution perturbations, enabling explicit separation of aleatoric and epistemic uncertainty. Across five benchmarks—irregular spiral function approximation, Boston Housing and Kin8nm regression, ETTm1 and Jena-Climate forecasting, XJTU-SY/PRONOSTIA/HUST bearing prognostics, and Udacity plus OpenAI CarRacing control—NSAC is consistently competitive on accuracy and often stronger on uncertainty quality. On the spiral task it reaches MSE 0.0002 and CRPS 0.0095, outperforming Deep Ensembles and Monte-Carlo Dropout; on ETTm1 it achieves the best NLL at -1.3816 while maintaining reasonable forecast calibration; on XJTU-SY it obtains MSE 0.0048 and CRPS 0.0261. The key novelty is architectural: uncertainty is embedded inside continuous-time attention itself, with a closed-form forward pass that avoids numerical SDE solvers and remains interpretable at the neuronal cell level.