Revisiting Skin Tone Fairness in Dermatological Lesion Classification
Thorsten Kalb Kaisar Kushibar Celia Cintas Karim Lekadir Oliver Diaz Richard Osuala
Addressing fairness in lesion classification from dermatological images is
crucial due to variations in how skin diseases manifest across skin tones.
However, the absence of skin tone labels in public datasets hinders building a
fair classifier. To date, such skin tone labels have been estimated prior to
fairness analysis in independent studies using the Individual Typology Angle
(ITA). Briefly, ITA calculates an angle based on pixels extracted from skin
images taking into account the lightness and yellow-blue tints. These angles
are then categorised into skin tones that are subsequently used to analyse
fairness in skin cancer classification. In this work, we review and compare
four ITA-based approaches of skin tone classification on the ISIC18 dataset, a
common benchmark for assessing skin cancer classification fairness in the
literature. Our analyses reveal a high disagreement among previously published
studies demonstrating the risks of ITA-based skin tone estimation methods.
Moreover, we investigate the causes of such large discrepancy among these
approaches and find that the lack of diversity in the ISIC18 dataset limits its
use as a testbed for fairness analysis. Finally, we recommend further research
on robust ITA estimation and diverse dataset acquisition with skin tone
annotation to facilitate conclusive fairness assessments of artificial
intelligence tools in dermatology. Our code is available at
https://github.com/tkalbl/RevisitingSkinToneFairness.