Revealing Vulnerabilities in Stable Diffusion via Targeted Attacks
Chenyu Zhang Lanjun Wang Anan Liu
Recent developments in text-to-image models, particularly Stable Diffusion,
have marked significant achievements in various applications. With these
advancements, there are growing safety concerns about the vulnerability of the
model that malicious entities exploit to generate targeted harmful images.
However, the existing methods in the vulnerability of the model mainly evaluate
the alignment between the prompt and generated images, but fall short in
revealing the vulnerability associated with targeted image generation. In this
study, we formulate the problem of targeted adversarial attack on Stable
Diffusion and propose a framework to generate adversarial prompts.
Specifically, we design a gradient-based embedding optimization method to craft
reliable adversarial prompts that guide stable diffusion to generate specific
images. Furthermore, after obtaining successful adversarial prompts, we reveal
the mechanisms that cause the vulnerability of the model. Extensive experiments
on two targeted attack tasks demonstrate the effectiveness of our method in
targeted attacks. The code can be obtained in
https://github.com/datar001/Revealing-Vulnerabilities-in-Stable-Diffusion-via-Targeted-Attacks.