Red AI? Inconsistent Responses from GPT3.5 Models on Political Issues in
  the US and China
Di Zhou Yinxian Zhang
The rising popularity of ChatGPT and other AI-powered large language models
(LLMs) has led to increasing studies highlighting their susceptibility to
mistakes and biases. However, most of these studies focus on models trained on
English texts. Taking an innovative approach, this study investigates political
biases in GPT's multilingual models. We posed the same question about
high-profile political issues in the United States and China to GPT in both
English and simplified Chinese, and our analysis of the bilingual responses
revealed that GPT's bilingual models' political "knowledge" (content) and the
political "attitude" (sentiment) are significantly more inconsistent on
political issues in China. The simplified Chinese GPT models not only tended to
provide pro-China information but also presented the least negative sentiment
towards China's problems, whereas the English GPT was significantly more
negative towards China. This disparity may stem from Chinese state censorship
and US-China geopolitical tensions, which influence the training corpora of GPT
bilingual models. Moreover, both Chinese and English models tended to be less
critical towards the issues of "their own" represented by the language used,
than the issues of "the other." This suggests that GPT multilingual models
could potentially develop a "political identity" and an associated sentiment
bias based on their training language. We discussed the implications of our
findings for information transmission and communication in an increasingly
divided world.