NTH

GOFish: Full Field-of-View Gaussian Splatting and Mesh Extraction

AuthorsAllen Tu, Meher Gitika Karumuri, Shifan Zhu, Hailing Wang, Yen-Liang Lin, David Held, Matthias Zwicker, Tom Goldstein

AffiliationsAmazon University of Maryland, College Park University of Massachusetts Amherst 4 5 Northeastern University Carnegie Mellon University https://3dgofish.github.io

November 13, 2026 2 min read
Watch on YouTube
The one-line take

GOFish turns ultra-wide fisheye images into real-time-renderable 3D scenes with collision meshes for simulation.

Key results

2.28
FIORD PSNR gain

dB improvement over the best prior method on the 200° FIORD benchmark.

1.97
FullCircle PSNR gain

dB improvement over the best prior method on the 205° FullCircle benchmark.

3.17
Wide-angle appearance PSNR gain

dB improvement over PPISP on FIORD.

0.249
FIORD mesh F-score

Score achieved with the optional Geometry Objective.

79
Real-time rendering speed

FPS on an NVIDIA A10G GPU.

5.4
Maximum Gaussian-count reduction

Fold reduction from Calibrated Mip Filtering in the reported ablation.

What the paper found

GOFish tackles a key limitation of Gaussian Splatting: existing fisheye methods miss parts of very wide views, while Gaussian mesh-reconstruction methods generally require rectified, pinhole images. In a project involving Amazon, the system trains directly on calibrated fisheye images and combines a calibration-defined ray domain, complete Gaussian-to-tile admission, and per-pixel ray-distance sorting to render beyond 180° and recover peripheral geometry. Calibrated densification builds out the surface, while Calibrated Mip Filtering and Wide-Angle Appearance Modeling—an extension of PPISP—keep pixel blur and lens falloff out of the shared scene and mesh. On FIORD’s 200° views, GOFish improves PSNR by 2.28 dB over the best prior method; on FullCircle’s 205° views, the gain is 1.97 dB. Its wide-angle appearance model alone adds 3.17 dB over PPISP on FIORD. With the optional Geometry Objective, its FIORD mesh reaches an F-score of 0.249. On an NVIDIA A10G, the real-time renderer reaches 79 FPS, and calibrated filtering cuts the Gaussian count by as much as 5.4× in the reported ablation. The result is a single scene representation that supports full-field rendering, ray tracing, and mesh-based collision geometry for simulation.

Original abstract

Robots, autonomous vehicles, and consumer 360° cameras use fisheye lenses for wide fields of view (FoVs). Simulation-ready digital twins require photorealistic rendering across the full FoV and an aligned collision mesh. However, prior fisheye Gaussian Splatting methods only rasterize up to 180° and do not produce meshes, while Gaussian surface reconstruction methods assume pinhole training images. We present GOFish, the first 3D Gaussian Splatting surface reconstruction method to train directly on fisheye images, rasterize FoVs beyond 180°, and extract an aligned mesh from the same scene. Calibrated densification recovers the surface across the entire FoV, while Calibrated Mip Filtering and Wide-Angle Appearance Modeling keep pixel blur and lens falloff out of the scene and mesh. Across the 200° FIORD and 205° FullCircle benchmarks, GOFish improves PSNR over the best prior method by 2.28 and 1.97 dB, respectively. It also outperforms fisheye rendering baselines within their own FoVs and achieves the highest mesh F-score on FIORD and the main ScanNet++ set. Together, the Gaussian scene and mesh form a LiDAR-free, simulation-ready digital twin that renders in real time, ray traces the full FoV, and provides collision geometry.

Read the original paper

More in Computer Vision

Browse all 82 papers →