TacBPM: A Tactile-conditioned Behavior Prior Model for Dexterous Reorientation

TacBPM is a tactile-conditioned behavior prior for dexterous object reorientation, addressing a weakness of direct joint-space reinforcement learning used in earlier systems, including OpenAI’s dexterous manipulation work: contact-rich finger gaits are difficult to rediscover when object geometry or sensing changes. The method trains 8 sphere specialists with PPO in Isaac Sim and Isaac Lab, covering scale factors from 0.3 to 1.0, then distills them into a variational latent controller. Its 192-dimensional tactile-proprioceptive history conditions a 16-D Gaussian latent prior, while a decoder converts residual latent commands into a 22-DoF hand action. During downstream learning, the prior remains frozen and PPO explores around its contact-stable latent mean, with decoder finetuning adapting behavior to new geometries. On unseen sphere scales and non-spherical objects, TacBPM reaches 60.38% average success, compared with 40.05% for the no-tactile variant; on the mixed Multi reorientation benchmark, it achieves 70.00% success versus 1.00% for raw-action PPO. The same paradigm extends to arm-hand Grasp-to-AnyPose: on the unseen staples marker, success rises from 0.49% with raw-action PPO to 97.36% with TacBPM. Six-axis simulation, 20-hertz hardware control, and qualitative sim-to-real demonstrations show that tactile conditioning helps select contact-aware rolling, regrasping, and stabilization strategies, although command-switch transients and substantial geometry shifts remain failure modes.