UniSHARP: Universal Sharp Monocular View Synthesis

UniSHARP extends the SHARP feedforward 3D Gaussian splatting framework into a universal monocular view synthesis system that works across perspective, wide-FoV, fisheye, and 360-degree panoramic cameras. The core technical move is a unified ray-distance space, where Gaussian primitives are anchored along predicted rays and radial distances rather than image-plane coordinates, allowing the model to transfer across heterogeneous projection models without camera-specific branches. It combines Geometry Anchored Gaussians with Feature Conditioned Gaussian residuals, fusing 2D semantic encodings and 3D ray-based features, and adds panoramic-specific regularization such as spherical Gaussian initialization and distortion-aware probabilistic dropout. To evaluate this setting, the authors build a FoV-stratified benchmark spanning 36,873 perspective, 10,692 wide-FoV, 14,163 fisheye, and 42,754 panoramic validation pairs. On this benchmark, UniSHARP reports strong gains over prior monocular baselines: on HM3D panoramas it reaches 29.244 PSNR, 0.895 SSIM, and 0.065 LPIPS, compared with Matrix3D at 23.398 PSNR; on OmniRooms-Wide it achieves 25.243 PSNR and 0.076 LPIPS; and on ScanNet++ Fisheye it reaches 20.660 PSNR and 0.184 LPIPS. The pose-free variant also remains competitive, and inference is efficient at 3.1 seconds per sample, much faster than PanoDreamer at 8.6 seconds and Matrix3D at 38.8 seconds.