The Spectral Neuron

The Spectral Neuron proposes a middle ground between transparent linear models and opaque neural networks. Instead of multiplying features by scalar weights, it forms an affine matrix pencil, A(x) = A₀ + ΣxᵢAᵢ, with learned real symmetric matrices, and uses one eigenvalue, λₖ(A(x)), as the prediction. This creates explicit, controllable nonlinearity: the smallest eigenvalue is concave, the largest is convex, positive-semidefinite coefficient matrices enforce monotonicity, and the spectral norm of each Aᵢ provides a global bound on that feature’s influence. Eigenvectors and eigenspaces also yield signed local influence measures and tighter instance-specific bounds, including at repeated eigenvalues through Clarke subdifferentials. Internal eigenvalues can universally approximate continuous functions on compact domains, while increasing matrix dimension increases expressive power without changing the model’s interpretable structure. Training uses standard eigensolvers, automatic differentiation in PyTorch, and Adam; the proposed initialization maintains eigenvalue gaps and avoids simultaneous diagonalization, which would reduce the model to piecewise-linear behavior. Experiments show scaling benefits on synthetic functions, the Criteo Display Advertising Challenge with 45,840,617 rows, and HIGGS with 11,000,000 rows; a 15-dimensional neuron fit bivariate targets of complexity 13 quite well, while monotonicity improved data efficiency on shape-constrained tasks. The approach is computationally expensive because dense eigendecomposition scales cubically with matrix dimension, so it targets applications where coefficient transparency and guaranteed shape constraints matter more than state-of-the-art accuracy. The implementation was assisted by OpenAI Codex.