Coupling-Robust Accuracy in Multiphysics Physics Informed Neural Networks via Kronecker-Preconditioned Optimization
AuthorsYoungjae Park, Jaemin Kim, Junghwa Hong
Resources
This paper shows that a smarter preconditioned optimizer can make physics-informed neural networks far more stable when solving tightly coupled multiphysics problems.
Key results
A 234-experiment factorial comparison across 2 optimizers, 3 balancing schemes, and 4 coupled multiphysics systems.
Three 1D benchmark systems were used: thermoelasticity, reaction–diffusion, and Nernst–Planck–Poisson.
The strongest benchmark was a 2D electroosmotic flow system that directly resolves the electric double layer with 6 PDEs.
At strongest coupling, SOAP improved final L2 error by 143× over Adam on the thermoelasticity benchmark.
SOAP+GN achieved relative L2 of 1.3×10^-3 on the 2D, 6-PDE electroosmotic flow problem at ε = 0.2.
Adam+GN failed on the 2D electroosmotic flow benchmark with final L2 above 0.9.
What the paper found
This paper explains why physics-informed neural networks for coupled multiphysics systems lose accuracy as coupling strengthens, and shows how Kronecker-preconditioned optimization fixes it. For linearly coupled PDEs, the authors prove that the standard neural tangent kernel, K = JJ⊤, has spectral radius growing as Ω(γ²), where γ is the coupling strength, so the stable learning rate shrinks as O(1/γ²). In contrast, block-diagonal Gauss–Newton preconditioning yields a preconditioned kernel KP that decomposes into orthogonal projectors and satisfies λmax(KP) ≤ S, with S equal to the number of networks, independent of γ; in the overparameterized regime they observe equality λmax(KP) = S across all tests. They connect this theory to SOAP, a Kronecker-factored optimizer, and inverse-gradient-norm balancing, reporting that SOAP+GN is the only method to keep coupling degradation near 1.0× across 234 experiments on 1D thermoelasticity, 1D reaction–diffusion, nonlinear Nernst–Planck–Poisson, and a 2D electroosmotic-flow benchmark. At strongest coupling, SOAP improves final L2 error by 41× to 143× over Adam, while SOAP+GN achieves 1.3×10⁻3 relative L2 on a 2D, 6-PDE electric-double-layer-resolved electroosmotic flow problem where Adam+GN fails with L2 above 0.9. The key novelty is the spectral argument: coupling hurts because diagonal optimizers cannot see cross-network Jacobian blocks, whereas blockwise second-order structure makes training coupling-robust.
Original abstract
Physics-informed neural networks (PINNs) for coupled multiphysics systems suffer systematic accuracy degradation as inter-equation coupling strengthens. We provide a theoretical explanation for this phenomenon through neural tangent kernel (NTK) analysis: for linearly coupled systems, we prove that the standard NTK's spectral radius grows as $Ω(γ^2)$ with coupling strength $γ$, shrinking the stable learning rate, while block-diagonal Gauss--Newton (GN) preconditioning yields a preconditioned NTK $K_P = J H^{+} J^\top$ (where $H$ is the block-diagonal GN Hessian) whose spectral radius is bounded by $S$ ($S$ = number of networks), independent of $γ$. We verify the $Ω(γ^2)$ growth numerically across symmetric, asymmetric, and nonlinear coupled PDE systems, and confirm $λ_{\max}(K_P) = S$ with equality in all cases. Combining the Kronecker-preconditioned optimizer SOAP with inverse-gradient-norm loss balancing (SOAP+GN) yields coupling-robust accuracy: across 234 experiments spanning three 1D systems of increasing nonlinearity and a 2D electroosmotic flow benchmark, SOAP+GN maintains final-epoch $L_2$ degradation $\leq 1.1\times$ (ratio of strong- to weak-coupling error) even as coupling parameters vary over one to two orders of magnitude, compared with $> 10^2\times$ for Adam+GN. SOAP+GN further scales to a 2D, 6-PDE electroosmotic flow system at EDL-resolved conditions -- a regime that all prior PINN electrokinetics studies have avoided through simplified physics -- where Adam+GN fails entirely ($L_2 > 0.9$).
Read the original paperMore in Optimization
Browse all 36 papers →An $Ω(κ_y^8ε^{-6})$ Lower Bound for Stochastic NC-SC Bilevel Optimization with First-order Oracles
Zhihao Gu, Qilong Wu, Junchi Yang
This work proves that stochastic bilevel optimization fundamentally requires up to epsilon^{-6} oracle queries, showing existing methods are asymptotically optimal.
Hyper Algorithm Design Agent: Evolving Learnable Optimizer from Zero
Zipei Yu, Yue-Jiao Gong, Zeyuan Ma, Yuncheng Jiang, Zhiguang Cao
A pair of self-improving coding agents evolves new learnable optimization algorithms from a simple template, reducing the need for handcrafted optimizer design.
Tight Regret Bound for Online Inverse Linear Optimization via Multiscale Matrix Weights
Shinsaku Sakaue
A new multiscale matrix-weights algorithm learns hidden linear preferences online with provably optimal dimension-dependent regret.