Uncovering Safety Risks in Open-source LLMs through Concept Activation
  Vector
Zhihao Xu Ruixuan Huang Xiting Wang Fangzhao Wu Jing Yao Xing Xie
Current open-source large language models (LLMs) are often undergone careful
safety alignment before public release. Some attack methods have also been
proposed that help check for safety vulnerabilities in LLMs to ensure alignment
robustness. However, many of these methods have moderate attack success rates.
Even when successful, the harmfulness of their outputs cannot be guaranteed,
leading to suspicions that these methods have not accurately identified the
safety vulnerabilities of LLMs. In this paper, we introduce a LLM attack method
utilizing concept-based model explanation, where we extract safety concept
activation vectors (SCAVs) from LLMs' activation space, enabling efficient
attacks on well-aligned LLMs like LLaMA-2, achieving near 100% attack success
rate as if LLMs are completely unaligned. This suggests that LLMs, even after
thorough safety alignment, could still pose potential risks to society upon
public release. To evaluate the harmfulness of outputs resulting with various
attack methods, we propose a comprehensive evaluation method that reduces the
potential inaccuracies of existing evaluations, and further validate that our
method causes more harmful content. Additionally, we discover that the SCAVs
show some transferability across different open-source LLMs.