Multi-task learning with cross-task consistency for improved depth
  estimation in colonoscopy
Pedro Esteban Chavarrias Solano Andrew Bulpitt Venkataraman Subramanian Sharib Ali
Colonoscopy screening is the gold standard procedure for assessing
abnormalities in the colon and rectum, such as ulcers and cancerous polyps.
Measuring the abnormal mucosal area and its 3D reconstruction can help quantify
the surveyed area and objectively evaluate disease burden. However, due to the
complex topology of these organs and variable physical conditions, for example,
lighting, large homogeneous texture, and image modality estimating distance
from the camera aka depth) is highly challenging. Moreover, most colonoscopic
video acquisition is monocular, making the depth estimation a non-trivial
problem. While methods in computer vision for depth estimation have been
proposed and advanced on natural scene datasets, the efficacy of these
techniques has not been widely quantified on colonoscopy datasets. As the
colonic mucosa has several low-texture regions that are not well pronounced,
learning representations from an auxiliary task can improve salient feature
extraction, allowing estimation of accurate camera depths. In this work, we
propose to develop a novel multi-task learning (MTL) approach with a shared
encoder and two decoders, namely a surface normal decoder and a depth estimator
decoder. Our depth estimator incorporates attention mechanisms to enhance
global context awareness. We leverage the surface normal prediction to improve
geometric feature extraction. Also, we apply a cross-task consistency loss
among the two geometrically related tasks, surface normal and camera depth. We
demonstrate an improvement of 14.17% on relative error and 10.4% improvement on
$\delta_{1}$ accuracy over the most accurate baseline state-of-the-art BTS
approach. All experiments are conducted on a recently released C3VD dataset;
thus, we provide a first benchmark of state-of-the-art methods.