Towards actionability for open medical imaging datasets: lessons from
  community-contributed platforms for data management and stewardship
Amelia Jiménez-Sánchez Natalia-Rozalia Avlona Dovile Juodelyte Théo Sourget Caroline Vang-Larsen Hubert Dariusz Zając Veronika Cheplygina
Medical imaging datasets are fundamental to artificial intelligence (AI) in
healthcare. The accuracy, robustness and fairness of diagnostic algorithms
depend on the data (and its quality) on which the models are trained and
evaluated. Medical imaging datasets have become increasingly available to the
public, and are often hosted on Community-Contributed Platforms (CCP),
including private companies like Kaggle or HuggingFace. While open data is
important to enhance the redistribution of data's public value, we find that
the current CCP governance model fails to uphold the quality needed and
recommended practices for sharing, documenting, and evaluating datasets. In
this paper we investigate medical imaging datasets on CCPs and how they are
documented, shared, and maintained. We first highlight some differences between
medical imaging and computer vision, particularly in the potentially harmful
downstream effects due to poor adoption of recommended dataset management
practices. We then analyze 20 (10 medical and 10 computer vision) popular
datasets on CCPs and find vague licenses, lack of persistent identifiers and
storage, duplicates and missing metadata, with differences between the
platforms. We present "actionability" as a conceptual metric to reveal the data
quality gap between characteristics of data on CCPs and the desired
characteristics of data for AI in healthcare. Finally, we propose a
commons-based stewardship model for documenting, sharing and maintaining
datasets on CCPs and end with a discussion of limitations and open questions.