Trade-Offs Between Fairness and Privacy in Language Modeling
Cleo Matzken Steffen Eger Ivan Habernal
Protecting privacy in contemporary NLP models is gaining in importance. So
does the need to mitigate social biases of such models. But can we have both at
the same time? Existing research suggests that privacy preservation comes at
the price of worsening biases in classification tasks. In this paper, we
explore the extent to which this tradeoff really holds when we incorporate both
privacy preservation and de-biasing techniques into training text generation
models. How does improving the model along one dimension affect the other
dimension as well as the utility of the model? We conduct an extensive set of
experiments that include bias detection, privacy attacks, language modeling,
and performance on downstream tasks.