Towards detecting unanticipated bias in Large Language Models
Anna Kruspe
Over the last year, Large Language Models (LLMs) like ChatGPT have become
widely available and have exhibited fairness issues similar to those in
previous machine learning systems. Current research is primarily focused on
analyzing and quantifying these biases in training data and their impact on the
decisions of these models, alongside developing mitigation strategies. This
research largely targets well-known biases related to gender, race, ethnicity,
and language. However, it is clear that LLMs are also affected by other, less
obvious implicit biases. The complex and often opaque nature of these models
makes detecting such biases challenging, yet this is crucial due to their
potential negative impact in various applications. In this paper, we explore
new avenues for detecting these unanticipated biases in LLMs, focusing
specifically on Uncertainty Quantification and Explainable AI methods. These
approaches aim to assess the certainty of model decisions and to make the
internal decision-making processes of LLMs more transparent, thereby
identifying and understanding biases that are not immediately apparent. Through
this research, we aim to contribute to the development of fairer and more
transparent AI systems.