MetricTrail: Revisiting the Power of Metric Depth and Local Optimization for Long-Sequence Streaming 3D Reconstruction
AuthorsJingnan Gao, Jiahui Huang, Sven Elflein, Jianhao Zheng, Bingxin Ke, Xiaokang Yang, Yichao Yan, Laura Leal-Taixé, Qunjie Zhou, Jiawei Ren, Shengyu Huang
AffiliationsNVIDIA · Shanghai Jiao Tong University · University of Toronto, Vector Institute · Stanford University · Equal advising https://research.nvidia.com/labs/sil/projects/metrictrail/
MetricTrail uses metric depth and DINOv3 features to make streaming 3D reconstruction from long videos more accurate and efficient.
Key results
Meters; full-sequence trajectory error.
Reduction relative to LingBot-MAP, whose average ATE is 5.15 m.
Meters; full-sequence trajectory error.
Meters; dense reconstruction.
FPS with supplied intrinsics on an NVIDIA A100.
GB with supplied intrinsics.
What the paper found
MetricTrail from NVIDIA strengthens DPVO’s bounded local optimization for long-video 3D reconstruction without widening its optimization window. It initializes sparse patch depths with MoGe-2 metric predictions and keeps them as robust soft constraints during bundle adjustment, anchoring scale while allowing multi-view evidence to correct unreliable depths. Frozen DINOv3-S/16 features, fused through learned adapters, improve patch matching and recurrent context; adaptation uses short clips rather than long-sequence training. On Oxford Spires, MetricTrail reaches 1.77 m average trajectory error, 65.6% lower than LingBot-MAP’s 5.15 m. With loop closure, its VBR trajectory error is 10.88 m. Dense reconstruction records 0.04 m Chamfer distance on 7Scenes. With supplied intrinsics on KITTI-02, the system runs at 12.70 FPS with 3.21 GB peak GPU memory on an NVIDIA A100. It also supports calibrated stereo and independently replaceable depth providers.
Original abstract
We present MetricTrail, a streaming system for accurate camera motion estimation and dense reconstruction from long monocular videos. To keep inference efficient over long sequences, we revisit DPVO's sparse local estimator. We find that single-image metric depth substantially improves its trajectory accuracy, without retraining the correspondence network or enlarging the optimization window. Metric predictions initialize patch depths and remain as soft constraints during optimization. We further improve correspondence estimation through learned adapters that augment matching features and recurrent context with frozen DINOv3 representations. The correspondence estimator and adapters are trained on diverse short clips without additional long-sequence training. This modular design also allows the depth provider to be independently upgraded and supports supplied camera intrinsics and calibrated stereo observations without retraining. Under the reported RGB-only protocols, MetricTrail achieves the lowest average trajectory error among evaluated methods on KITTI, Oxford Spires, and VBR. Pairing the estimated poses with single-image depth predictions also produces strong dense reconstruction on indoor and outdoor benchmarks and coherent reconstructions of in-the-wild videos. With supplied intrinsics, the system processes KITTI-02 at 12.70 FPS with 3.21 GB peak GPU memory.
Read the original paperMore in Computer Vision
Browse all 82 papers →InstantHMR : Real-Time Human Mesh Recovery on Your Phone
Mohamed Adjel, Kahina Chalabi, Vincent Bonnet
InstantHMR reconstructs detailed 3D human meshes from images in milliseconds, including on a smartphone.
Stochastic Deep Supervision for Gallery-Consistent Early-Exit Face Recognition
Guray Ozgur, Tahar Chettaoui, Eduarda Caldeira, Naser Damer, Fadi Boutros
SD-ViT trains every ViT depth to produce useful face embeddings, enabling cheaper early-exit recognition without changing the gallery.
VAE Rhythm: Temporal Decoding Traces for AI-Generated Video Detection
Ke Sun, Shuwei Li, Haiyang Mei, Mike Zheng Shou
A periodic trace left by video-generation VAEs could help spot AI-made videos even when their visible artifacts are gone.