On Prediction-Modelers and Decision-Makers: Why Fairness Requires More
  Than a Fair Prediction Model
Teresa Scantamburlo Joachim Baumann Christoph Heitz
An implicit ambiguity in the field of prediction-based decision-making
regards the relation between the concepts of prediction and decision. Much of
the literature in the field tends to blur the boundaries between the two
concepts and often simply speaks of 'fair prediction.' In this paper, we point
out that a differentiation of these concepts is helpful when implementing
algorithmic fairness. Even if fairness properties are related to the features
of the used prediction model, what is more properly called 'fair' or 'unfair'
is a decision system, not a prediction model. This is because fairness is about
the consequences on human lives, created by a decision, not by a prediction. We
clarify the distinction between the concepts of prediction and decision and
show the different ways in which these two elements influence the final
fairness properties of a prediction-based decision system. In addition to
exploring this relationship conceptually and practically, we propose a
framework that enables a better understanding and reasoning of the conceptual
logic of creating fairness in prediction-based decision-making. In our
framework, we specify different roles, namely the 'prediction-modeler' and the
'decision-maker,' and the information required from each of them for being able
to implement fairness of the system. Our framework allows for deriving distinct
responsibilities for both roles and discussing some insights related to ethical
and legal requirements. Our contribution is twofold. First, we shift the focus
from abstract algorithmic fairness to context-dependent decision-making,
recognizing diverse actors with unique objectives and independent actions.
Second, we provide a conceptual framework that can help structure
prediction-based decision problems with respect to fairness issues, identify
responsibilities, and implement fairness governance mechanisms in real-world
scenarios.