BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction
  using Neural Radiance Fields
Shreya Saha Sainan Liu Shan Lin Jingpei Lu Michael Yip
Reconstruction of deformable scenes from endoscopic videos is important for
many applications such as intraoperative navigation, surgical visual
perception, and robotic surgery. It is a foundational requirement for realizing
autonomous robotic interventions for minimally invasive surgery. However,
previous approaches in this domain have been limited by their modular nature
and are confined to specific camera and scene settings. Our work adopts the
Neural Radiance Fields (NeRF) approach to learning 3D implicit representations
of scenes that are both dynamic and deformable over time, and furthermore with
unknown camera poses. We demonstrate this approach on endoscopic surgical
scenes from robotic surgery. This work removes the constraints of known camera
poses and overcomes the drawbacks of the state-of-the-art unstructured dynamic
scene reconstruction technique, which relies on the static part of the scene
for accurate reconstruction. Through several experimental datasets, we
demonstrate the versatility of our proposed model to adapt to diverse camera
and scene settings, and show its promise for both current and future robotic
surgical systems.