GOFish:Abstract
Robots, autonomous vehicles, and consumer 360° cameras use fisheye lenses for wide fields of view (FoVs). Simulation-ready digital twins require photorealistic rendering across the full FoV and an aligned collision mesh. However, prior fisheye Gaussian Splatting methods only rasterize up to 180° and do not produce meshes, while Gaussian surface reconstruction methods assume pinhole training images.
We present GOFish, the first 3D Gaussian Splatting surface reconstruction method to train directly on fisheye images, rasterize FoVs beyond 180°, and extract an aligned mesh from the same scene. Calibrated densification recovers the surface across the entire FoV, while Calibrated Mip Filtering and Wide-Angle Appearance Modeling keep pixel blur and lens falloff out of the scene and mesh.
Across the 200° FIORD and 205° FullCircle benchmarks, GOFish improves PSNR over the best prior method by 2.28 and 1.97 dB, respectively. It also outperforms fisheye rendering baselines within their own FoVs and achieves the highest mesh F-score on FIORD and the main ScanNet++ set. Together, the Gaussian scene and mesh form a LiDAR-free, simulation-ready digital twin that renders in real time, ray traces the full FoV, and provides collision geometry.
Method
3DGUT [31] has two wide-angle rasterization failures: (a) it crops the field of view (FoV) to 160° and sorts by optical-axis depth, which reverses beyond 90°, and (b) it approximates tile coverage with projected sigma points even though rendering uses the exact ray response. Together, they produce the cutoff ring and tile-shaped artifacts shown below.
GOFish addresses these failures by letting calibration define the valid image domain. Complete admission draws every Gaussian reached by a valid ray, while sorting each pixel by ray distance preserves near-to-far compositing beyond 90°. Calibrated densification measures gradients in image space, allowing one threshold to recover detail across the full FoV.
Calibrated Mip Filtering (above) uses the camera Jacobian to separate a view-independent scene filter from each view’s pixel footprint. Applying that footprint only during rendering keeps view-specific blur out of the scene and mesh and keeps the densified scene compact. At inference, faster projected tile selection enables real-time rendering.
Wide-Angle Appearance Modeling
Exposure, vignetting, and rim falloff belong to the camera rather than the 3D scene. GOFish’s Wide-Angle Appearance Modeling extends PPISP [4] with a per-camera attenuation that combines a monotone radial profile and low-resolution flat field to capture the steep rim falloff missed by its low-order vignetting. This prevents rim darkening from being baked into Gaussian colors and inherited by the mesh, improving FIORD PSNR by 3.17 dB over standard PPISP.
Full Field-of-View Mesh Extraction
GOFish renders the same ray-maximum response used by Gaussian Opacity Fields (GOF) [37], enabling direct opacity-field mesh extraction from the Gaussian scene. Full-FoV coverage ensures that points seen only by peripheral rays are evaluated rather than treated as unobserved. The optional Geometry Objective further improves mesh quality.
Together, the Gaussian scene and aligned mesh form a simulation-ready digital twin with photorealistic full-FoV rendering, ray tracing, and collision geometry.
BibTeX
@misc{Tu2026GOFish,
author = {Tu, Allen and Karumuri, Meher Gitika and Zhu, Shifan and Wang, Hailing and Lin, Yen-Liang and Held, David and Zwicker, Matthias and Goldstein, Tom},
title = {{GOFish}: Full Field-of-View Gaussian Splatting and Mesh Extraction},
year = {2026},
eprint = {2610.37818},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.37818}
}