Do Moral Judgment and Reasoning Capability of LLMs Change with Language?
  A Study using the Multilingual Defining Issues Test
Aditi Khandelwal Utkarsh Agarwal Kumar Tanmay Monojit Choudhury
This paper explores the moral judgment and moral reasoning abilities
exhibited by Large Language Models (LLMs) across languages through the Defining
Issues Test. It is a well known fact that moral judgment depends on the
language in which the question is asked. We extend the work of beyond English,
to 5 new languages (Chinese, Hindi, Russian, Spanish and Swahili), and probe
three LLMs -- ChatGPT, GPT-4 and Llama2Chat-70B -- that shows substantial
multilingual text processing and generation abilities. Our study shows that the
moral reasoning ability for all models, as indicated by the post-conventional
score, is substantially inferior for Hindi and Swahili, compared to Spanish,
Russian, Chinese and English, while there is no clear trend for the performance
of the latter four languages. The moral judgments too vary considerably by the
language.