BiasAsker: Measuring the Bias in Conversational AI System
Yuxuan Wan Wenxuan Wang Pinjia He Jiazhen Gu Haonan Bai Michael Lyu
Powered by advanced Artificial Intelligence (AI) techniques, conversational
AI systems, such as ChatGPT and digital assistants like Siri, have been widely
deployed in daily life. However, such systems may still produce content
containing biases and stereotypes, causing potential social problems. Due to
the data-driven, black-box nature of modern AI techniques, comprehensively
identifying and measuring biases in conversational systems remains a
challenging task. Particularly, it is hard to generate inputs that can
comprehensively trigger potential bias due to the lack of data containing both
social groups as well as biased properties. In addition, modern conversational
systems can produce diverse responses (e.g., chatting and explanation), which
makes existing bias detection methods simply based on the sentiment and the
toxicity hardly being adopted. In this paper, we propose BiasAsker, an
automated framework to identify and measure social bias in conversational AI
systems. To obtain social groups and biased properties, we construct a
comprehensive social bias dataset, containing a total of 841 groups and 8,110
biased properties. Given the dataset, BiasAsker automatically generates
questions and adopts a novel method based on existence measurement to identify
two types of biases (i.e., absolute bias and related bias) in conversational
systems. Extensive experiments on 8 commercial systems and 2 famous research
models, such as ChatGPT and GPT-3, show that 32.83% of the questions generated
by BiasAsker can trigger biased behaviors in these widely deployed
conversational systems. All the code, data, and experimental results have been
released to facilitate future research.