Query-Efficient Decision-based Black-Box Patch Attack
Zhaoyu Chen Bo Li Shuang Wu Shouhong Ding Wenqiang Zhang
Deep neural networks (DNNs) have been showed to be highly vulnerable to
imperceptible adversarial perturbations. As a complementary type of adversary,
patch attacks that introduce perceptible perturbations to the images have
attracted the interest of researchers. Existing patch attacks rely on the
architecture of the model or the probabilities of predictions and perform
poorly in the decision-based setting, which can still construct a perturbation
with the minimal information exposed -- the top-1 predicted label. In this
work, we first explore the decision-based patch attack. To enhance the attack
efficiency, we model the patches using paired key-points and use targeted
images as the initialization of patches, and parameter optimizations are all
performed on the integer domain. Then, we propose a differential evolutionary
algorithm named DevoPatch for query-efficient decision-based patch attacks.
Experiments demonstrate that DevoPatch outperforms the state-of-the-art
black-box patch attacks in terms of patch area and attack success rate within a
given query budget on image classification and face verification. Additionally,
we conduct the vulnerability evaluation of ViT and MLP on image classification
in the decision-based patch attack setting for the first time. Using DevoPatch,
we can evaluate the robustness of models to black-box patch attacks. We believe
this method could inspire the design and deployment of robust vision models
based on various DNN architectures in the future.