Leveraging Near-Field Lighting for Monocular Depth Estimation from
  Endoscopy Videos
Akshay Paruchuri Samuel Ehrenstein Shuxian Wang Inbar Fried Stephen M. Pizer Marc Niethammer Roni Sengupta
Monocular depth estimation in endoscopy videos can enable assistive and
robotic surgery to obtain better coverage of the organ and detection of various
health issues. Despite promising progress on mainstream, natural image depth
estimation, techniques perform poorly on endoscopy images due to a lack of
strong geometric features and challenging illumination effects. In this paper,
we utilize the photometric cues, i.e., the light emitted from an endoscope and
reflected by the surface, to improve monocular depth estimation. We first
create two novel loss functions with supervised and self-supervised variants
that utilize a per-pixel shading representation. We then propose a novel depth
refinement network (PPSNet) that leverages the same per-pixel shading
representation. Finally, we introduce teacher-student transfer learning to
produce better depth maps from both synthetic data with supervision and
clinical data with self-supervision. We achieve state-of-the-art results on the
C3VD dataset while estimating high-quality depth maps from clinical data. Our
code, pre-trained models, and supplementary materials can be found on our
project page: https://ppsnet.github.io/