Bypassing the Safety Training of Open-Source LLMs with Priming Attacks
Jason Vega Isha Chaudhary Changming Xu Gagandeep Singh
With the recent surge in popularity of LLMs has come an ever-increasing need
for LLM safety training. In this paper, we show that SOTA open-source LLMs are
vulnerable to simple, optimization-free attacks we refer to as $\textit{priming
attacks}$, which are easy to execute and effectively bypass alignment from
safety training. Our proposed attack improves the Attack Success Rate on
Harmful Behaviors, as measured by Llama Guard, by up to $3.3\times$ compared to
baselines. Source code and data are available at
https://github.com/uiuc-focal-lab/llm-priming-attacks .