How User Language Affects Conflict Fatality Estimates in ChatGPT
Daniel Kazenwadel Christoph V. Steinert
OpenAI's ChatGPT language model has gained popularity as a powerful tool for
complex problem-solving and information retrieval. However, concerns arise
about the reproduction of biases present in the language-specific training
data. In this study, we address this issue in the context of the
Israeli-Palestinian and Turkish-Kurdish conflicts. Using GPT-3.5, we employed
an automated query procedure to inquire about casualties in specific
airstrikes, in both Hebrew and Arabic for the former conflict and Turkish and
Kurdish for the latter. Our analysis reveals that GPT-3.5 provides 27$\pm$11
percent lower fatality estimates when queried in the language of the attacker
than in the language of the targeted group. Evasive answers denying the
existence of such attacks further increase the discrepancy, creating a novel
bias mechanism not present in regular search engines. This language bias has
the potential to amplify existing media biases and contribute to information
bubbles, ultimately reinforcing conflicts.