Inference for an Algorithmic Fairness-Accuracy Frontier
Yiqi Liu Francesca Molinari
Decision-making processes increasingly rely on the use of algorithms. Yet,
algorithms' predictive ability frequently exhibit systematic variation across
subgroups of the population. While both fairness and accuracy are desirable
properties of an algorithm, they often come at the cost of one another. What
should a fairness-minded policymaker do then, when confronted with finite data?
In this paper, we provide a consistent estimator for a theoretical
fairness-accuracy frontier put forward by Liang, Lu and Mu (2023) and propose
inference methods to test hypotheses that have received much attention in the
fairness literature, such as (i) whether fully excluding a covariate from use
in training the algorithm is optimal and (ii) whether there are less
discriminatory alternatives to an existing algorithm. We also provide an
estimator for the distance between a given algorithm and the fairest point on
the frontier, and characterize its asymptotic distribution. We leverage the
fact that the fairness-accuracy frontier is part of the boundary of a convex
set that can be fully represented by its support function. We show that the
estimated support function converges to a tight Gaussian process as the sample
size increases, and then express policy-relevant hypotheses as restrictions on
the support function to construct valid test statistics.