Towards unraveling calibration biases in medical image analysis
María Agustina Ricci Lara Candelaria Mosquera Enzo Ferrante Rodrigo Echeveste
In recent years the development of artificial intelligence (AI) systems for
automated medical image analysis has gained enormous momentum. At the same
time, a large body of work has shown that AI systems can systematically and
unfairly discriminate against certain populations in various application
scenarios. These two facts have motivated the emergence of algorithmic fairness
studies in this field. Most research on healthcare algorithmic fairness to date
has focused on the assessment of biases in terms of classical discrimination
metrics such as AUC and accuracy. Potential biases in terms of model
calibration, however, have only recently begun to be evaluated. This is
especially important when working with clinical decision support systems, as
predictive uncertainty is key for health professionals to optimally evaluate
and combine multiple sources of information. In this work we study
discrimination and calibration biases in models trained for automatic detection
of malignant dermatological conditions from skin lesions images. Importantly,
we show how several typically employed calibration metrics are systematically
biased with respect to sample sizes, and how this can lead to erroneous
fairness analysis if not taken into consideration. This is of particular
relevance to fairness studies, where data imbalance results in drastic sample
size differences between demographic sub-groups, which, if not taken into
account, can act as confounders.