Improving Robustness for Vision Transformer with a Simple Dynamic
  Scanning Augmentation
Shashank Kotyan Danilo Vasconcellos Vargas
Vision Transformer (ViT) has demonstrated promising performance in computer
vision tasks, comparable to state-of-the-art neural networks. Yet, this new
type of deep neural network architecture is vulnerable to adversarial attacks
limiting its capabilities in terms of robustness. This article presents a novel
contribution aimed at further improving the accuracy and robustness of ViT,
particularly in the face of adversarial attacks. We propose an augmentation
technique called `Dynamic Scanning Augmentation' that leverages dynamic input
sequences to adaptively focus on different patches, thereby maintaining
performance and robustness. Our detailed investigations reveal that this
adaptability to the input sequence induces significant changes in the attention
mechanism of ViT, even for the same image. We introduce four variations of
Dynamic Scanning Augmentation, outperforming ViT in terms of both robustness to
adversarial attacks and accuracy against natural images, with one variant
showing comparable results. By integrating our augmentation technique, we
observe a substantial increase in ViT's robustness, improving it from $17\%$ to
$92\%$ measured across different types of adversarial attacks. These findings,
together with other comprehensive tests, indicate that Dynamic Scanning
Augmentation enhances accuracy and robustness by promoting a more adaptive type
of attention. In conclusion, this work contributes to the ongoing research on
Vision Transformers by introducing Dynamic Scanning Augmentation as a technique
for improving the accuracy and robustness of ViT. The observed results
highlight the potential of this approach in advancing computer vision tasks and
merit further exploration in future studies.