Non-discrimination Criteria for Generative Language Models
Sara Sterlie Nina Weng Aasa Feragen
Within recent years, generative AI, such as large language models, has
undergone rapid development. As these models become increasingly available to
the public, concerns arise about perpetuating and amplifying harmful biases in
applications. Gender stereotypes can be harmful and limiting for the
individuals they target, whether they consist of misrepresentation or
discrimination. Recognizing gender bias as a pervasive societal construct, this
paper studies how to uncover and quantify the presence of gender biases in
generative language models. In particular, we derive generative AI analogues of
three well-known non-discrimination criteria from classification, namely
independence, separation and sufficiency. To demonstrate these criteria in
action, we design prompts for each of the criteria with a focus on occupational
gender stereotype, specifically utilizing the medical test to introduce the
ground truth in the generative AI context. Our results address the presence of
occupational gender bias within such conversational language models.