Investigating the Quality of DermaMNIST and Fitzpatrick17k
  Dermatological Image Datasets
Kumar Abhishek Aditi Jain Ghassan Hamarneh
The remarkable progress of deep learning in dermatological tasks has brought
us closer to achieving diagnostic accuracies comparable to those of human
experts. However, while large datasets play a crucial role in the development
of reliable deep neural network models, the quality of data therein and their
correct usage are of paramount importance. Several factors can impact data
quality, such as the presence of duplicates, data leakage across train-test
partitions, mislabeled images, and the absence of a well-defined test
partition. In this paper, we conduct meticulous analyses of two popular
dermatological image datasets: DermaMNIST and Fitzpatrick17k, uncovering these
data quality issues, measure the effects of these problems on the benchmark
results, and propose corrections to the datasets. Besides ensuring the
reproducibility of our analysis, by making our analysis pipeline and the
accompanying code publicly available, we aim to encourage similar explorations
and to facilitate the identification and addressing of potential data quality
issues in other large datasets.