Debiasing Sentence Embedders through Contrastive Word Pairs
Philip Kenneweg Sarah Schröder Alexander Schulz Barbara Hammer
Over the last years, various sentence embedders have been an integral part in
the success of current machine learning approaches to Natural Language
Processing (NLP). Unfortunately, multiple sources have shown that the bias,
inherent in the datasets upon which these embedding methods are trained, is
learned by them. A variety of different approaches to remove biases in
embeddings exists in the literature. Most of these approaches are applicable to
word embeddings and in fewer cases to sentence embeddings. It is problematic
that most debiasing approaches are directly transferred from word embeddings,
therefore these approaches fail to take into account the nonlinear nature of
sentence embedders and the embeddings they produce. It has been shown in
literature that bias information is still present if sentence embeddings are
debiased using such methods. In this contribution, we explore an approach to
remove linear and nonlinear bias information for NLP solutions, without
impacting downstream performance. We compare our approach to common debiasing
methods on classical bias metrics and on bias metrics which take nonlinear
information into account.