DESTEIN: Navigating Detoxification of Language Models via Universal
  Steering Pairs and Head-wise Activation Fusion
Yu Li Zhihua Wei Han Jiang Chuanyang Gong
Despite the remarkable achievements of language models (LMs) across a broad
spectrum of tasks, their propensity for generating toxic outputs remains a
prevalent concern. Current solutions involving fine-tuning or auxiliary models
usually require extensive memory and computational resources, rendering them
less practical for deployment in large language models (LLMs). In this paper,
we propose DeStein, a novel method that detoxififies LMs by altering their
internal representations in the activation space with lower resource and time
cost. Specifically, we leverage self-induced steering pairs to identify
detoxification vectors through arithmetic operations in the activation space.
During inference, detoxification is achieved by blending the detoxification
vectors with the original representations. Empirical results demonstrate that
our method significantly outperforms previous state-of-the-art approaches on
popular detoxification metrics, while also maintaining satisfactory generation
quality and diversity. Furthermore, we extend our method to multiple LLMs,
demonstrating its practicality and scalability. Warning: some example model
outputs contain highly offensive or disturbing text.