Auditing and Mitigating Cultural Bias in LLMs
Yan Tao Olga Viberg Ryan S. Baker Rene F. Kizilcec
Culture fundamentally shapes people's reasoning, behavior, and communication.
Generative artificial intelligence (AI) technologies may cause a shift towards
a dominant culture. As people increasingly use AI to expedite and even automate
various professional and personal tasks, cultural values embedded in AI models
may bias authentic expression. We audit large language models for cultural
bias, comparing their responses to nationally representative survey data, and
evaluate country-specific prompting as a mitigation strategy. We find that
GPT-4, 3.5 and 3 exhibit cultural values resembling English-speaking and
Protestant European countries. Our mitigation strategy reduces cultural bias in
recent models but not for all countries/territories. To avoid cultural bias in
generative AI, especially in high-stakes contexts, we suggest using culture
matching and ongoing cultural audits.