Can ultrasound confidence maps predict sonographers' labeling
  variability?
Vanessa Gonzalez Duque Leonhard Zirus Yordanka Velikova Nassir Navab Diana Mateus
Measuring cross-sectional areas in ultrasound images is a standard tool to
evaluate disease progress or treatment response. Often addressed today with
supervised deep-learning segmentation approaches, existing solutions highly
depend upon the quality of experts' annotations. However, the annotation
quality in ultrasound is anisotropic and position-variant due to the inherent
physical imaging principles, including attenuation, shadows, and missing
boundaries, commonly exacerbated with depth. This work proposes a novel
approach that guides ultrasound segmentation networks to account for
sonographers' uncertainties and generate predictions with variability similar
to the experts. We claim that realistic variability can reduce overconfident
predictions and improve physicians' acceptance of deep-learning cross-sectional
segmentation solutions. Our method provides CM's certainty for each pixel for
minimal computational overhead as it can be precalculated directly from the
image. We show that there is a correlation between low values in the confidence
maps and expert's label uncertainty. Therefore, we propose to give the
confidence maps as additional information to the networks. We study the effect
of the proposed use of ultrasound CMs in combination with four state-of-the-art
neural networks and in two configurations: as a second input channel and as
part of the loss. We evaluate our method on 3D ultrasound datasets of the
thyroid and lower limb muscles. Our results show ultrasound CMs increase the
Dice score, improve the Hausdorff and Average Surface Distances, and decrease
the number of isolated pixel predictions. Furthermore, our findings suggest
that ultrasound CMs improve the penalization of uncertain areas in the ground
truth data, thereby improving problematic interpolations. Our code and example
data will be made public at
https://github.com/IFL-CAMP/Confidence-segmentation.