Leveraging a realistic synthetic database to learn Shape-from-Shading
  for estimating the colon depth in colonoscopy images
Josué Ruano Martín Gómez Eduardo Romero Antoine Manzanera
Colonoscopy is the choice procedure to diagnose colon and rectum cancer, from
early detection of small precancerous lesions (polyps), to confirmation of
malign masses. However, the high variability of the organ appearance and the
complex shape of both the colon wall and structures of interest make this
exploration difficult. Learned visuospatial and perceptual abilities mitigate
technical limitations in clinical practice by proper estimation of the
intestinal depth. This work introduces a novel methodology to estimate colon
depth maps in single frames from monocular colonoscopy videos. The generated
depth map is inferred from the shading variation of the colon wall with respect
to the light source, as learned from a realistic synthetic database. Briefly, a
classic convolutional neural network architecture is trained from scratch to
estimate the depth map, improving sharp depth estimations in haustral folds and
polyps by a custom loss function that minimizes the estimation error in edges
and curvatures. The network was trained by a custom synthetic colonoscopy
database herein constructed and released, composed of 248,400 frames (47
videos), with depth annotations at the level of pixels. This collection
comprehends 5 subsets of videos with progressively higher levels of visual
complexity. Evaluation of the depth estimation with the synthetic database
reached a threshold accuracy of 95.65%, and a mean-RMSE of 0.451 cm, while a
qualitative assessment with a real database showed consistent depth
estimations, visually evaluated by the expert gastroenterologist coauthoring
this paper. Finally, the method achieved competitive performance with respect
to another state-of-the-art method using a public synthetic database and
comparable results in a set of images with other five state-of-the-art methods.