New Job, New Gender? Measuring the Social Bias in Image Generation
  Models
Wenxuan Wang Haonan Bai Jen-tse Huang Yuxuan Wan Youliang Yuan Haoyi Qiu Nanyun Peng Michael R. Lyu
Image generation models can generate or edit images from a given text. Recent
advancements in image generation technology, exemplified by DALL-E and
Midjourney, have been groundbreaking. These advanced models, despite their
impressive capabilities, are often trained on massive Internet datasets, making
them susceptible to generating content that perpetuates social stereotypes and
biases, which can lead to severe consequences. Prior research on assessing bias
within image generation models suffers from several shortcomings, including
limited accuracy, reliance on extensive human labor, and lack of comprehensive
analysis. In this paper, we propose BiasPainter, a novel metamorphic testing
framework that can accurately, automatically and comprehensively trigger social
bias in image generation models. BiasPainter uses a diverse range of seed
images of individuals and prompts the image generation models to edit these
images using gender, race, and age-neutral queries. These queries span 62
professions, 39 activities, 57 types of objects, and 70 personality traits. The
framework then compares the edited images to the original seed images, focusing
on any changes related to gender, race, and age. BiasPainter adopts a testing
oracle that these characteristics should not be modified when subjected to
neutral prompts. Built upon this design, BiasPainter can trigger the social
bias and evaluate the fairness of image generation models. To evaluate the
effectiveness of BiasPainter, we use BiasPainter to test five widely-used
commercial image generation software and models, such as stable diffusion and
Midjourney. Experimental results show that 100\% of the generated test cases
can successfully trigger social bias in image generation models.