Automatic Construction of a Korean Toxic Instruction Dataset for Ethical
  Tuning of Large Language Models
Sungjoo Byun Dongjun Jang Hyemi Jo Hyopil Shin
Caution: this paper may include material that could be offensive or
distressing.
  The advent of Large Language Models (LLMs) necessitates the development of
training approaches that mitigate the generation of unethical language and
aptly manage toxic user queries. Given the challenges related to human labor
and the scarcity of data, we present KoTox, comprising 39K unethical
instruction-output pairs. This collection of automatically generated toxic
instructions refines the training of LLMs and establishes a foundational
framework for improving LLMs' ethical awareness and response to various toxic
inputs, promoting more secure and responsible interactions in Natural Language
Processing (NLP) applications.