CULTURE-GEN: Revealing Global Cultural Perception in Language Models
  through Natural Language Prompting
Huihan Li Liwei Jiang Nouha Dziri Xiang Ren Yejin Choi
As the utilization of large language models (LLMs) has proliferated
worldwide, it is crucial for them to have adequate knowledge and fair
representation for diverse global cultures. In this work, we uncover culture
perceptions of three SOTA models on 110 countries and regions on 8
culture-related topics through culture-conditioned generations, and extract
symbols from these generations that are associated to each culture by the LLM.
We discover that culture-conditioned generation consist of linguistic "markers"
that distinguish marginalized cultures apart from default cultures. We also
discover that LLMs have an uneven degree of diversity in the culture symbols,
and that cultures from different geographic regions have different presence in
LLMs' culture-agnostic generation. Our findings promote further research in
studying the knowledge and fairness of global culture perception in LLMs. Code
and Data can be found in: https://github.com/huihanlhh/Culture-Gen/