Measuring Intersectional Biases in Historical Documents
Nadav Borenstein Karolina Stańczak Thea Rolskov Natália da Silva Perez Natacha Klein Käfer Isabelle Augenstein
Data-driven analyses of biases in historical texts can help illuminate the
origin and development of biases prevailing in modern society.
  However, digitised historical documents pose a challenge for NLP
practitioners as these corpora suffer from errors introduced by optical
character recognition (OCR) and are written in an archaic language. In this
paper, we investigate the continuities and transformations of bias in
historical newspapers published in the Caribbean during the colonial era (18th
to 19th centuries). Our analyses are performed along the axes of gender, race,
and their intersection. We examine these biases by conducting a temporal study
in which we measure the development of lexical associations using
distributional semantics models and word embeddings. Further, we evaluate the
effectiveness of techniques designed to process OCR-generated data and assess
their stability when trained on and applied to the noisy historical newspapers.
We find that there is a trade-off between the stability of the word embeddings
and their compatibility with the historical dataset. We provide evidence that
gender and racial biases are interdependent, and their intersection triggers
distinct effects. These findings align with the theory of intersectionality,
which stresses that biases affecting people with multiple marginalised
identities compound to more than the sum of their constituents.