ZipSplat: Fewer Gaussians, Better Splats

ZipSplat, from ETH Zürich and Microsoft with Marc Pollefeys as an author, is a feed-forward 3D Gaussian Splatting method that breaks the usual one-Gaussian-per-pixel design by decoding compact scene tokens into unconstrained 3D Gaussians. A multi-view backbone, instantiated with DA3-Giant, produces visual tokens that are compressed by k-means into scene tokens, refined by cross- and self-attention, and decoded by a lightweight MLP into 32 Gaussians per token; a one-directional Chamfer geometric loss, depth supervision, coupled initialization, and a progressive view/compression schedule stabilize free 3D placement. On DL3DV and RealEstate10K, ZipSplat reaches state-of-the-art pose-free novel view synthesis while using about 6× fewer Gaussians than pixel-aligned baselines, including a 24× reduction versus DA3 on DL3DV. It improves over YoNoSplat by 2.1 dB PSNR on DL3DV and 1.2 dB on RealEstate10K, and with test-time token optimization it adds about 5 dB more PSNR in roughly 3 seconds on a single RTX 4090. The model also generalizes zero-shot to Mip-NeRF360 and ScanNet++, where its pose-free variant reaches 21.72 and 18.01 PSNR at 32 views, respectively, and a pose-conditioned version climbs to 23.49 PSNR on ScanNet++. Because compression is set at inference, one trained model traces a continuous quality-efficiency curve: at 192 views, scaled compression reduces storage from about 183 MB to 9.3 MB and boosts rendering speed from 40 FPS to 401 FPS.