GULP3R: Gaussian Underwater spLatting with semantic Perception for 3D Reconstruction. Creating Labeled 3D Maps from Camera Images

Completed - 2026 | Python - SfM (COLMAP) - Image Enhancement - Underwater Perception - SAM3 - Gaussian Splatting


How can we create labeled high-resolution 3D maps of the underwater world to help scientists explore and understand our oceans, using nothing but a sequence of images?

This is the question I explored this summer while living a long-time dream: getting involved in ocean research at one of the world’s leading marine research institutes, the Monterey Bay Aquarium Research Institute (MBARI). I had the opportunity to join the CoMPAS Lab as a summer intern, working under the amazing mentorship of Pushyami Kaveti and Giancarlo Troni.

To achieve our objective, of creating labeled 3D Maps from camera images, we propose and evaluate a custom pipeline, GULP3R, built with open-source algorithms for image enhancement, 3D reconstruction, segmentation, and semantic Gaussian Splatting. We evaluate GULP3R on field datasets acquired in Monterey Bay and the Arctic Ocean using Remotely Operated Vehicles (ROVs) from the MBARI fleet (MiniROV, ROV Ventana, MOLA AUV). Validation on our datasets shows quantitatively promising results on 2D novel-view synthesis, and qualitatively cleaner 3D geometric and semantic reconstruction, as well as rendered 2D semantic masks. Quantitative evaluation of 3D metrics and semantics remains future work, due to the lack of ground truth.


GULP3R Pipeline developed from state-of-the art perception algorithms.
(3.1) SfM : Process input images to estimate the scene as point cloud and cameras’ extrinsics/intrinsics.
(3.2) Color Enhance : If necessary, enhance color and re-run SfM.
(3.3) Semantic Segment: Extract segmentation masks.
(3.4) Semantic Lift: Project semantics onto the sparse point cloud.
(3.5) Semantic Splat: Learn a semantic Gaussian cloud.

I have been able to collect one of the tested dataset onboard of the R/V Rachel Carson, using the scientific forward looking camera of the ROV Ventana! You can read more about my personal experience at MBARI during the 10 weeks of internship in my Blog Post! Or from the SciComm Web Story!


Photo credits: MBARI’s SciComm Team, Rose Dalrymple © 2026