White Men Lead, Black Women Help: Uncovering Gender, Racial, and
  Intersectional Bias in Language Agency
Yixin Wan Kai-Wei Chang
Social biases can manifest in language agency. For instance, White
individuals and men are often described as "agentic" and achievement-oriented,
whereas Black individuals and women are frequently described as "communal" and
as assisting roles. This study establishes agency as an important aspect of
studying social biases in both human-written and Large Language Model
(LLM)-generated texts. To accurately measure "language agency" at sentence
level, we propose a Language Agency Classification dataset to train reliable
agency classifiers. We then use an agency classifier to reveal notable language
agency biases in 6 datasets of human- or LLM-written texts, including
biographies, professor reviews, and reference letters. While most prior NLP
research on agency biases focused on single dimensions, we comprehensively
explore language agency biases in gender, race, and intersectional identities.
We observe that (1) language agency biases in human-written texts align with
real-world social observations; (2) LLM-generated texts demonstrate remarkably
higher levels of language agency bias than human-written texts; and (3)
critical biases in language agency target people of minority groups -- for
instance, languages used to describe Black females exhibit the lowest level of
agency across datasets. Our findings reveal intricate social biases in human-
and LLM-written texts through the lens of language agency, warning against
using LLM generations in social contexts without scrutiny.