CLoSeR: Closing the Loop for Long-Context Streaming Reconstruction
AuthorsMoyang Li, Zihan Zhu, Wei Zhang, Marc Pollefeys, Daniel Barath
AffiliationsETH Zurich · University of Stuttgart · Microsoft
CLoSeR helps 3D reconstruction models stay accurate over kilometer-long streams by detecting revisited locations and correcting accumulated pose drift.
Key results
Meters; compared with 36.16 m for LoGeR.
Meters; compared with 25.44 m for LoGeR.
Meters.
Meters.
Frames per second.
What the paper found
CLoSeR adds explicit loop closure to LoGeR, a streaming model for reconstructing 3D scenes from uncalibrated monocular video. It retrieves revisit candidates using SALAD image descriptors, then builds loop-conditioned windows pairing frames from the current and earlier parts of the stream. LoGeR estimates their relative poses in a shared coordinate frame; CLoSeR combines those loop constraints with sequential constraints in a robust SE(3) pose-graph optimization, using a Huber loss to reduce the effect of bad matches. Because LoGeR maintains globally consistent scale, CLoSeR avoids the more complex Sim(3) or SL(4) optimization used by submap-based methods. On VBR, its average trajectory error is 11.15 m, compared with 36.16 m for LoGeR; on KITTI, the error is 12.79 m versus 25.44 m. CLoSeR also reaches 0.75 m on DROID-W and 5.26 m on Oxford Spires, outperforming the other feedforward methods reported on those benchmarks. On an NVIDIA RTX 4090, it runs at 7.12 frames per second. The method does not require retraining, but it cannot correct a complete tracking failure in its underlying frontend.
Original abstract
Feedforward foundation models have recently shown remarkable 3D reconstruction capabilities. However, existing models exhibit large tracking drift in long-context streaming reconstruction due to error accumulation. In this paper, we revisit loop closure with streaming reconstruction foundation models to enable accurate, drift-free, kilometer-scale reconstruction. Specifically, our method detects loop candidates through global descriptor retrieval, and constructs loop-conditioned windows to estimate the relative poses between looped frames. Given the observation that our adopted streaming reconstruction backbone produces a globally consistent scale, we optimize all frame poses on the SE(3) manifold with sequential and loop closure constraints, avoiding the pose graph optimization on the Sim(3) or higher-dimensional SL(4) manifolds employed in prior works. Extensive experiments show that our method reduces drift and produces consistent geometry on kilometer-scale sequences, significantly outperforming the state of the art. Code is available at https://github.com/MoyangLi00/CLoSeR.git.
Read the original paperMore in Computer Vision
Browse all 67 papers →CompactSplat: Spatially Adaptive Gaussian Distribution for Feedforward Scene Reconstruction
Bing He, Jingnan Gao, Yunuo Chen, Zhengxue Cheng, Rong Xie, Li Song, Wenjun Zhang
CompactSplat reconstructs 3D scenes by placing Gaussian primitives densely where detail matters and sparsely across flat regions.
SpaceFlow: Locally Controllable 3D Generation
Neil De La Fuente, Joan Lafuente, Mukhammadali Sayfiddinov, Felicia Scharitzer, Marc Pollefeys, Ata Celen, Sayan Deb Sarkar, Elisabetta Fedele
SpaceFlow lets creators control how closely individual 3D regions follow supplied shapes and independently guide each part’s appearance.
LEGO: A Lifting-Free Approach for Exocentric-to-Egocentric Video Generation
Suhwan Cho, Yonwoo Choi, Soongjin Kim, Jicheol Park, Taegyu Lim
LEGO replaces brittle depth-based camera lifting with a learned transformer view synthesizer that gives diffusion models better structure for generating first-person video.