Unleashing the Strengths of Unlabeled Data in Pan-cancer Abdominal Organ
  Quantification: the FLARE22 Challenge
Jun Ma Yao Zhang Song Gu Cheng Ge Shihao Ma Adamo Young Cheng Zhu Kangkang Meng Xin Yang Ziyan Huang Fan Zhang Wentao Liu YuanKe Pan Shoujin Huang Jiacheng Wang Mingze Sun Weixin Xu Dengqiang Jia Jae Won Choi Natália Alves Bram de Wilde Gregor Koehler Yajun Wu Manuel Wiesenfarth Qiongjie Zhu Guoqiang Dong Jian He the FLARE Challenge Consortium Bo Wang
Quantitative organ assessment is an essential step in automated abdominal
disease diagnosis and treatment planning. Artificial intelligence (AI) has
shown great potential to automatize this process. However, most existing AI
algorithms rely on many expert annotations and lack a comprehensive evaluation
of accuracy and efficiency in real-world multinational settings. To overcome
these limitations, we organized the FLARE 2022 Challenge, the largest abdominal
organ analysis challenge to date, to benchmark fast, low-resource, accurate,
annotation-efficient, and generalized AI algorithms. We constructed an
intercontinental and multinational dataset from more than 50 medical groups,
including Computed Tomography (CT) scans with different races, diseases,
phases, and manufacturers. We independently validated that a set of AI
algorithms achieved a median Dice Similarity Coefficient (DSC) of 90.0\% by
using 50 labeled scans and 2000 unlabeled scans, which can significantly reduce
annotation requirements. The best-performing algorithms successfully
generalized to holdout external validation sets, achieving a median DSC of
89.5\%, 90.9\%, and 88.3\% on North American, European, and Asian cohorts,
respectively. They also enabled automatic extraction of key organ biology
features, which was labor-intensive with traditional manual measurements. This
opens the potential to use unlabeled data to boost performance and alleviate
annotation shortages for modern AI models.