Building Socio-culturally Inclusive Stereotype Resources with Community
  Engagement
Sunipa Dev Jaya Goyal Dinesh Tewari Shachi Dave Vinodkumar Prabhakaran
With rapid development and deployment of generative language models in global
settings, there is an urgent need to also scale our measurements of harm, not
just in the number and types of harms covered, but also how well they account
for local cultural contexts, including marginalized identities and the social
biases experienced by them. Current evaluation paradigms are limited in their
abilities to address this, as they are not representative of diverse, locally
situated but global, socio-cultural perspectives. It is imperative that our
evaluation resources are enhanced and calibrated by including people and
experiences from different cultures and societies worldwide, in order to
prevent gross underestimations or skews in measurements of harm. In this work,
we demonstrate a socio-culturally aware expansion of evaluation resources in
the Indian societal context, specifically for the harm of stereotyping. We
devise a community engaged effort to build a resource which contains
stereotypes for axes of disparity that are uniquely present in India. The
resultant resource increases the number of stereotypes known for and in the
Indian context by over 1000 stereotypes across many unique identities. We also
demonstrate the utility and effectiveness of such expanded resources for
evaluations of language models. CONTENT WARNING: This paper contains examples
of stereotypes that may be offensive.