DSAP: Analyzing Bias Through Demographic Comparison of Datasets
Iris Dominguez-Catena Daniel Paternain Mikel Galar
In the last few years, Artificial Intelligence systems have become
increasingly widespread. Unfortunately, these systems can share many biases
with human decision-making, including demographic biases. Often, these biases
can be traced back to the data used for training, where large uncurated
datasets have become the norm. Despite our knowledge of these biases, we still
lack general tools to detect and quantify them, as well as to compare the
biases in different datasets. Thus, in this work, we propose DSAP (Demographic
Similarity from Auxiliary Profiles), a two-step methodology for comparing the
demographic composition of two datasets. DSAP can be deployed in three key
applications: to detect and characterize demographic blind spots and bias
issues across datasets, to measure dataset demographic bias in single datasets,
and to measure dataset demographic shift in deployment scenarios. An essential
feature of DSAP is its ability to robustly analyze datasets without explicit
demographic labels, offering simplicity and interpretability for a wide range
of situations. To show the usefulness of the proposed methodology, we consider
the Facial Expression Recognition task, where demographic bias has previously
been found. The three applications are studied over a set of twenty datasets
with varying properties. The code is available at
https://github.com/irisdominguez/DSAP.