NeSAM: Neuro-Symbolic Kinodynamics with Soil Adaptation for Off-Road Mobility
AuthorsChenhui Pan, Tong Xu, Francesco Cancelliere, Xuesu Xiao
Resources
NeSAM helps off-road robots predict and navigate deformable terrain by blending soil physics with learned dynamics that adapt online.
Key results
Maximum prediction-accuracy improvement over the strongest compared baselines in simulation.
Maximum prediction-accuracy improvement on real-world vehicle data.
Hausdorff-distance reduction from online EKF soil adaptation in closed-loop simulation.
Approximate number of simulated interaction transitions used for training and evaluation.
Prediction horizon in seconds for the 32-step rollout at 10 Hz.
What the paper found
NeSAM is a neuro-symbolic kinodynamic model for predicting six-degree-of-freedom off-road vehicle motion on deformable terrain. It combines learned elevation and semantic terrain representations with differentiable Bekker-Wong terramechanics, which models soil-dependent sinkage, shear, traction, and contact forces, while a Transformer predicts wheel sinkage and a chassis-level residual correction. Newton-Euler dynamics then produces the structured vehicle-state update. During deployment, an extended Kalman filter, or EKF, estimates interpretable soil-parameter corrections from discrepancies between predicted and measured velocities, enabling online adaptation without changing the learned network. Trained on 125000 Verti-Bench transitions and tested in 32-step, 3.2-second autoregressive rollouts, NeSAM improves prediction accuracy by up to 30% in simulation and 29% on physical Verti-4-Wheeler data relative to the strongest baselines. When coupled with an MPPI navigation controller under out-of-distribution soil conditions, online adaptation reduces Hausdorff trajectory error by 69.4%, from 6.87 meters to 2.10 meters, demonstrating more reliable tracking. The approach remains limited by the fixed constitutive assumptions of Bekker-Wong pressure-sinkage and shear models, which may not represent terrain behaviors that substantially violate those mechanics.
Original abstract
Accurate prediction of off-road vehicle motion over deformable terrain remains challenging because sinkage, slip, and traction vary with local soil conditions. Existing learning-based kinodynamic models directly approximate vehicle-terrain interactions from data but do not explicitly represent soil mechanics and offer limited physical interpretability. To address these limitations, we present NeSAM, a neuro-symbolic framework that combines differentiable Bekker-Wong terramechanics with learned terrain representations and a Transformer-based residual dynamics model for long-horizon, six degree-of-freedom kinodynamic prediction. The terramechanics component models soil-dependent interaction forces, while the residual model corrects discrepancies between the analytical prediction and the observed vehicle dynamics. NeSAM further estimates physically meaningful soil parameters from terrain observations and updates them online using an extended Kalman filter. We evaluate NeSAM in Verti-Bench, a simulator built on the Chrono multiphysics engine, and validate its performance on a physical Verti-4-Wheeler platform. NeSAM improves prediction accuracy by up to 30% in simulation and 29% on real-world data relative to the strongest compared baselines. When integrated with a close-loop navigation controller, NeSAM further improves traversal success rate through online soil adaptation while reduces Hausdorff distance to the reference trajectory by 69.4%, indicating improved trajectory tracking accuracy.
Read the original paperMore in Robotics
Browse all 50 papers →JAMB: Joint Action-Motion Diffusion for Bimanual Manipulation
Chuyang Xiao, Peilin Meng, David Held
JAMB helps two robot arms coordinate by jointly imagining their future movements and the changing 3D scene before acting.
Rolling-WAM: World Action Models with Rolling Imagination
Yinghua Zhou, Junjie Ye, Yiqi Zhao, Hao Dong, Celina Shiyu Wang, Ruohai Ge, Tingyi Yang, Basile Van Hoorick, Gaurav Sukhatme, Vitor Guizilini, Yue Wang
Rolling-WAM keeps future robot actions partially imagined and refined over time, making world-model-based manipulation replan 4.5 times faster.
Training-free Behavior Cloning
Maximilian Adang, Timothy Chen, Lars Osterberg, Aiden Swann, Mac Schwager
A fast, training-free robot controller reuses and corrects demonstration trajectories to deliver traceable behavior at real-time speeds.