InstantHMR : Real-Time Human Mesh Recovery on Your Phone
AuthorsMohamed Adjel, Kahina Chalabi, Vincent Bonnet
AffiliationsNaturalPad, Montpellier, France · LAAS-CNRS, Toulouse, France
InstantHMR reconstructs detailed 3D human meshes from images in milliseconds, including on a smartphone.
Key results
Milliseconds per person for InstantHMR-M without the optional fit.
Millimeters for InstantHMR-M without the fit.
Millimeters for InstantHMR-M without the fit.
Times faster on the phone with the one-crop fit, while matching NLF-S on EMDB-1 and 3DPW.
Person crops with released SAM 3D Body annotations.
What the paper found
InstantHMR targets a mobile bottleneck in single-image 3D human mesh recovery: accurate models often cannot run on a phone’s neural processor. It combines an HGNetV2 convolutional backbone with a four-layer transformer decoder to predict the Momentum Human Rig, trained on 4.2M person crops with released annotations from Meta’s SAM 3D Body dataset. On a Samsung Galaxy S23 with a Qualcomm Snapdragon 8 Gen 2 NPU, the medium model produces a 6,890-vertex mesh in 5.49 ms, with PA-MPJPE of 48.8 mm on EMDB-1 and 40.0 mm on 3DPW. Its key refinement is an optional, parameter-free Levenberg–Marquardt step: using an exact Jacobian and Newton–Schulz matrix iterations, it adjusts the pose to align the 3D body with the model’s own 2D keypoints, entirely inside the fp16 NPU graph. With one crop, this fit reaches 45.2 mm on EMDB-1 and 39.4 mm on 3DPW in 6.40 ms, matching NLF-S on those benchmarks while running 10.6× faster on the phone. The study also reports using Anthropic’s Claude Code to assist with evaluation tooling, not model training.
Original abstract
Real-time 3D human mesh recovery from a single image on mobile devices is a challenging problem with many applications, including robotics, biomechanics, AR/VR, and exergames. Current state-of-the-art models are either too slow to run efficiently on mobile devices or sacrifice accuracy for speed. We present InstantHMR, a family of open-source single-image regressors suited for mobile inference, based on the Momentum Human Rig (MHR) parametric model and trained exclusively on publicly available annotations from the SAM 3D Body dataset. Our approach achieves remarkable performance on a Samsung Galaxy S23 NPU, recovering a 6,890-vertex body mesh in 5.5 ms per person, and 0.98 ms on an NVIDIA RTX 4070 GPU. This speed surpasses all existing methods we evaluated. On mobile, it is 12x faster than the single mobile-compatible method that slightly outperforms ours in accuracy, while achieving competitive accuracy with 40.0 mm PA-MPJPE on 3DPW, 48.8 mm on EMDB and 41.7 mm on RICH. To ensure fair comparisons, we re-evaluate 11 state-of-the-art methods under a unified protocol aligned with the closest MHR-based competitors, including Meta's official MHR-to-SMPL conversion, and report their reproduction results. Additionally, we measure latency across eight different targets, ranging from desktop TensorRT to two phone NPUs. Code and models are available at https://github.com/Naturalpad-studio/InstantHMR.
Read the original paperMore in Computer Vision
Browse all 82 papers →Stochastic Deep Supervision for Gallery-Consistent Early-Exit Face Recognition
Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Naser Damer, Fadi Boutros
SD-ViT trains every ViT depth to produce useful face embeddings, enabling cheaper early-exit recognition without changing the gallery.
VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection
Ke Sun, Shuwei Li, Haiyang Mei, Mike Zheng Shou
A periodic trace left by video-generation VAEs could help spot AI-made videos even when their visible artifacts are gone.
CoLa3D: Composable Latent 3D Decomposition
Xudong Cai, Yongcai Wang, Deying Li, Chuanxia Zheng
CoLa3D turns a whole-scene representation into a set of coherent, reusable 3D object meshes.