Intriguing Properties of Diffusion Models: A Large-Scale Dataset for
  Evaluating Natural Attack Capability in Text-to-Image Generative Models
Takami Sato Justin Yue Nanze Chen Ningfei Wang Qi Alfred Chen
Denoising probabilistic diffusion models have shown breakthrough performance
that can generate more photo-realistic images or human-level illustrations than
the prior models such as GANs. This high image-generation capability has
stimulated the creation of many downstream applications in various areas.
However, we find that this technology is indeed a double-edged sword: We
identify a new type of attack, called the Natural Denoising Diffusion (NDD)
attack based on the finding that state-of-the-art deep neural network (DNN)
models still hold their prediction even if we intentionally remove their robust
features, which are essential to the human visual system (HVS), by text
prompts. The NDD attack can generate low-cost, model-agnostic, and
transferrable adversarial attacks by exploiting the natural attack capability
in diffusion models. Motivated by the finding, we construct a large-scale
dataset, Natural Denoising Diffusion Attack (NDDA) dataset, to systematically
evaluate the risk of the natural attack capability of diffusion models with
state-of-the-art text-to-image diffusion models. We evaluate the natural attack
capability by answering 6 research questions. Through a user study to confirm
the validity of the NDD attack, we find that the NDD attack can achieve an 88%
detection rate while being stealthy to 93% of human subjects. We also find that
the non-robust features embedded by diffusion models contribute to the natural
attack capability. To confirm the model-agnostic and transferrable attack
capability, we perform the NDD attack against an AD vehicle and find that 73%
of the physically printed attacks can be detected as a stop sign. We hope that
our study and dataset can help our community to be aware of the risk of
diffusion models and facilitate further research toward robust DNN models.