Tackling Bias in Pre-trained Language Models: Current Trends and
  Under-represented Societies
Vithya Yogarajan Gillian Dobbie Te Taka Keegan Rostam J. Neuwirth
The benefits and capabilities of pre-trained language models (LLMs) in
current and future innovations are vital to any society. However, introducing
and using LLMs comes with biases and discrimination, resulting in concerns
about equality, diversity and fairness, and must be addressed. While
understanding and acknowledging bias in LLMs and developing mitigation
strategies are crucial, the generalised assumptions towards societal needs can
result in disadvantages towards under-represented societies and indigenous
populations. Furthermore, the ongoing changes to actual and proposed amendments
to regulations and laws worldwide also impact research capabilities in tackling
the bias problem. This research presents a comprehensive survey synthesising
the current trends and limitations in techniques used for identifying and
mitigating bias in LLMs, where the overview of methods for tackling bias are
grouped into metrics, benchmark datasets, and mitigation strategies. The
importance and novelty of this survey are that it explores the perspective of
under-represented societies. We argue that current practices tackling the bias
problem cannot simply be 'plugged in' to address the needs of under-represented
societies. We use examples from New Zealand to present requirements for
adopting existing techniques to under-represented societies.