Measuring, Interpreting, and Improving Fairness of Algorithms using
  Causal Inference and Randomized Experiments
James Enouen Tianshu Sun Yan Liu
Algorithm fairness has become a central problem for the broad adoption of
artificial intelligence. Although the past decade has witnessed an explosion of
excellent work studying algorithm biases, achieving fairness in real-world AI
production systems has remained a challenging task. Most existing works fail to
excel in practical applications since either they have conflicting measurement
techniques and/ or heavy assumptions, or require code-access of the production
models, whereas real systems demand an easy-to-implement measurement framework
and a systematic way to correct the detected sources of bias.
  In this paper, we leverage recent advances in causal inference and
interpretable machine learning to present an algorithm-agnostic framework
(MIIF) to Measure, Interpret, and Improve the Fairness of an algorithmic
decision. We measure the algorithm bias using randomized experiments, which
enables the simultaneous measurement of disparate treatment, disparate impact,
and economic value. Furthermore, using modern interpretability techniques, we
develop an explainable machine learning model which accurately interprets and
distills the beliefs of a blackbox algorithm. Altogether, these techniques
create a simple and powerful toolset for studying algorithm fairness,
especially for understanding the cost of fairness in practical applications
like e-commerce and targeted advertising, where industry A/B testing is already
abundant.