T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image
  Generation
Jialu Wang Xinyue Gabby Liu Zonglin Di Yang Liu Xin Eric Wang
Warning: This paper contains several contents that may be toxic, harmful, or
offensive.
  In the last few years, text-to-image generative models have gained remarkable
success in generating images with unprecedented quality accompanied by a
breakthrough of inference speed. Despite their rapid progress, human biases
that manifest in the training examples, particularly with regard to common
stereotypical biases, like gender and skin tone, still have been found in these
generative models. In this work, we seek to measure more complex human biases
exist in the task of text-to-image generations. Inspired by the well-known
Implicit Association Test (IAT) from social psychology, we propose a novel
Text-to-Image Association Test (T2IAT) framework that quantifies the implicit
stereotypes between concepts and valence, and those in the images. We replicate
the previously documented bias tests on generative models, including morally
neutral tests on flowers and insects as well as demographic stereotypical tests
on diverse social attributes. The results of these experiments demonstrate the
presence of complex stereotypical behaviors in image generations.