A Chinese Dataset for Evaluating the Safeguards in Large Language Models
Yuxia Wang Zenan Zhai Haonan Li Xudong Han Lizhi Lin Zhenxuan Zhang Jingru Zhao Preslav Nakov Timothy Baldwin
Many studies have demonstrated that large language models (LLMs) can produce
harmful responses, exposing users to unexpected risks when LLMs are deployed.
Previous studies have proposed comprehensive taxonomies of the risks posed by
LLMs, as well as corresponding prompts that can be used to examine the safety
mechanisms of LLMs. However, the focus has been almost exclusively on English,
and little has been explored for other languages. Here we aim to bridge this
gap. We first introduce a dataset for the safety evaluation of Chinese LLMs,
and then extend it to two other scenarios that can be used to better identify
false negative and false positive examples in terms of risky prompt rejections.
We further present a set of fine-grained safety assessment criteria for each
risk type, facilitating both manual annotation and automatic evaluation in
terms of LLM response harmfulness. Our experiments on five LLMs show that
region-specific risks are the prevalent type of risk, presenting the major
issue with all Chinese LLMs we experimented with. Warning: this paper contains
example data that may be offensive, harmful, or biased.