Uncertainty-guided annotation enhances segmentation with the
  human-in-the-loop
Nadieh Khalili Joey Spronck Francesco Ciompi Jeroen van der Laak Geert Litjens
Deep learning algorithms, often critiqued for their 'black box' nature,
traditionally fall short in providing the necessary transparency for trusted
clinical use. This challenge is particularly evident when such models are
deployed in local hospitals, encountering out-of-domain distributions due to
varying imaging techniques and patient-specific pathologies. Yet, this
limitation offers a unique avenue for continual learning. The
Uncertainty-Guided Annotation (UGA) framework introduces a human-in-the-loop
approach, enabling AI to convey its uncertainties to clinicians, effectively
acting as an automated quality control mechanism. UGA eases this interaction by
quantifying uncertainty at the pixel level, thereby revealing the model's
limitations and opening the door for clinician-guided corrections. We evaluated
UGA on the Camelyon dataset for lymph node metastasis segmentation which
revealed that UGA improved the Dice coefficient (DC), from 0.66 to 0.76 by
adding 5 patches, and further to 0.84 with 10 patches. To foster broader
application and community contribution, we have made our code accessible at