SurgFM-SLAM: Robust Surgical Scene Reconstruction via 3D Foundation Adaptation Model

Surgical scene reconstruction is a critical prerequisite for intraoperative navigation in robotic surgery. Although deep neural networks (DNNs) have advanced surgical scene reconstruction, their performance degrades severely when confronted with texture-sparse biological tissues and dynamic illumination changes.

To this end, we develop an end-to-end SurgFM-SLAM framework for surgical scene reconstruction by exploring geometric representation priors of pretrained 3D foundation models (FMs) and SLAM. Specifically, we first design a frame-similarity sampling strategy to keep surgical scene consistency among sampled frames, and then develop a surgical foundation model (SurgFM) by employing a low-rank adaptation (LoRA) method to mine 3D strong structural geometry representations from 3D FMs with the aid of parameter-efficient finetuning techniques, aiming to effectively capture dynamic illumination conditions and informative textures from surgical environments.

Finally, we embed SurgFM into the SLAM backend to construct SurgFM-SLAM to perform robust surgical scene reconstruction in an end-to-end manner, including tracking, mapping, and relocalization. Extensive experiments on the SimCol dataset demonstrate that SurgFM-SLAM achieves competitive performance across depth estimation, camera pose estimation, and 3D reconstruction through comparisons to state-of-the-art methods. Additionally, zero-shot generalization tests on the C3VD and SCARED datasets manifest the generalization of SurgFM-SLAM.

Unlike previous efforts that predominantly rely on photometric or rendering consistency at the image level, we ground depth and pose estimation as integral parts of robust and learned geometric priors. The full pipeline of SurgFM-SLAM consists of three tightly coupled components:

(1) Frame-similarity sampling strategy. From a temporal sliding window, we select frame pairs with strong visual overlap to provide stable training signals. This ensures that SurgFM learns robust correspondences despite the rapid viewpoint changes, occlusions, and specular highlights typical of endoscopic video.

(2) SurgFM: Surgical Foundation Model. Building on a pre-trained 3D foundation model, SurgFM is adapted to the surgical domain by inserting low-rank adapters (LoRA) into the MLP layers of both encoder and decoder blocks. All backbone weights are kept frozen, while position embeddings, decoder embeddings, and prediction heads are unfrozen to capture illumination-invariant geometric features specific to endoscopic imagery, while preserving zero-shot generalization to unseen surgical data.

(3) SLAM backend integration. The robust two-view priors produced by SurgFM (pointmaps and feature descriptors) are integrated into a deep learning-based SLAM backend, which drives camera tracking, dense mapping, and relocalization. This yields globally consistent reconstructions of extended surgical sequences in real time, while the system supports both known and unknown camera intrinsics.

We evaluate SurgFM-SLAM on three datasets: SimCol (intra-domain training and evaluation), C3VD (zero-shot evaluation), and SCARED (zero-shot evaluation). With camera intrinsics, SurgFM-SLAM achieves superior depth estimation performance on both SimCol and C3VD, outperforming state-of-the-art baselines across most metrics.

Our method achieves an Abs Rel of 0.046 on SimCol and 0.083 on C3VD (zero-shot), demonstrating both strong intra-domain accuracy and robust cross-domain generalization. SurgFM-SLAM further delivers the lowest relative pose error on SimCol, while producing dense, structurally coherent 3D reconstructions across all three datasets.