The Best Defense is a Good Offense: Adversarial Augmentation against
  Adversarial Attacks
Iuri Frosio Jan Kautz
Many defenses against adversarial attacks (\eg robust classifiers,
randomization, or image purification) use countermeasures put to work only
after the attack has been crafted. We adopt a different perspective to
introduce $A^5$ (Adversarial Augmentation Against Adversarial Attacks), a novel
framework including the first certified preemptive defense against adversarial
attacks. The main idea is to craft a defensive perturbation to guarantee that
any attack (up to a given magnitude) towards the input in hand will fail. To
this aim, we leverage existing automatic perturbation analysis tools for neural
networks. We study the conditions to apply $A^5$ effectively, analyze the
importance of the robustness of the to-be-defended classifier, and inspect the
appearance of the robustified images. We show effective on-the-fly defensive
augmentation with a robustifier network that ignores the ground truth label,
and demonstrate the benefits of robustifier and classifier co-training. In our
tests, $A^5$ consistently beats state of the art certified defenses on MNIST,
CIFAR10, FashionMNIST and Tinyimagenet. We also show how to apply $A^5$ to
create certifiably robust physical objects. Our code at
https://github.com/NVlabs/A5 allows experimenting on a wide range of scenarios
beyond the man-in-the-middle attack tested here, including the case of physical
attacks.