DOEPatch: Dynamically Optimized Ensemble Model for Adversarial Patches
  Generation
Wenyi Tan Yang Li Chenxing Zhao Zhunga Liu Quan Pan
Object detection is a fundamental task in various applications ranging from
autonomous driving to intelligent security systems. However, recognition of a
person can be hindered when their clothing is decorated with carefully designed
graffiti patterns, leading to the failure of object detection. To achieve
greater attack potential against unknown black-box models, adversarial patches
capable of affecting the outputs of multiple-object detection models are
required. While ensemble models have proven effective, current research in the
field of object detection typically focuses on the simple fusion of the outputs
of all models, with limited attention being given to developing general
adversarial patches that can function effectively in the physical world. In
this paper, we introduce the concept of energy and treat the adversarial
patches generation process as an optimization of the adversarial patches to
minimize the total energy of the ``person'' category. Additionally, by adopting
adversarial training, we construct a dynamically optimized ensemble model.
During training, the weight parameters of the attacked target models are
adjusted to find the balance point at which the generated adversarial patches
can effectively attack all target models. We carried out six sets of
comparative experiments and tested our algorithm on five mainstream object
detection models. The adversarial patches generated by our algorithm can reduce
the recognition accuracy of YOLOv2 and YOLOv3 to 13.19\% and 29.20\%,
respectively. In addition, we conducted experiments to test the effectiveness
of T-shirts covered with our adversarial patches in the physical world and
could achieve that people are not recognized by the object detection model.
Finally, leveraging the Grad-CAM tool, we explored the attack mechanism of
adversarial patches from an energetic perspective.