Evaluating Large Language Models through Gender and Racial Stereotypes
Ananya Malik
Language Models have ushered a new age of AI gaining traction within the NLP
community as well as amongst the general population. AI's ability to make
predictions, generations and its applications in sensitive decision-making
scenarios, makes it even more important to study these models for possible
biases that may exist and that can be exaggerated. We conduct a quality
comparative study and establish a framework to evaluate language models under
the premise of two kinds of biases: gender and race, in a professional setting.
We find out that while gender bias has reduced immensely in newer models, as
compared to older ones, racial bias still exists.