COLosSAL: A Benchmark for Cold-start Active Learning for 3D Medical
  Image Segmentation
Han Liu Hao Li Xing Yao Yubo Fan Dewei Hu Benoit Dawant Vishwesh Nath Zhoubing Xu Ipek Oguz
Medical image segmentation is a critical task in medical image analysis. In
recent years, deep learning based approaches have shown exceptional performance
when trained on a fully-annotated dataset. However, data annotation is often a
significant bottleneck, especially for 3D medical images. Active learning (AL)
is a promising solution for efficient annotation but requires an initial set of
labeled samples to start active selection. When the entire data pool is
unlabeled, how do we select the samples to annotate as our initial set? This is
also known as the cold-start AL, which permits only one chance to request
annotations from experts without access to previously annotated data.
Cold-start AL is highly relevant in many practical scenarios but has been
under-explored, especially for 3D medical segmentation tasks requiring
substantial annotation effort. In this paper, we present a benchmark named
COLosSAL by evaluating six cold-start AL strategies on five 3D medical image
segmentation tasks from the public Medical Segmentation Decathlon collection.
We perform a thorough performance analysis and explore important open questions
for cold-start AL, such as the impact of budget on different strategies. Our
results show that cold-start AL is still an unsolved problem for 3D
segmentation tasks but some important trends have been observed. The code
repository, data partitions, and baseline results for the complete benchmark
are publicly available at https://github.com/MedICL-VU/COLosSAL.