Robust Overfitting Does Matter: Test-Time Adversarial Purification With
  FGSM
Linyu Tang Lei Zhang
Numerous studies have demonstrated the susceptibility of deep neural networks
(DNNs) to subtle adversarial perturbations, prompting the development of many
advanced adversarial defense methods aimed at mitigating adversarial attacks.
Current defense strategies usually train DNNs for a specific adversarial attack
method and can achieve good robustness in defense against this type of
adversarial attack. Nevertheless, when subjected to evaluations involving
unfamiliar attack modalities, empirical evidence reveals a pronounced
deterioration in the robustness of DNNs. Meanwhile, there is a trade-off
between the classification accuracy of clean examples and adversarial examples.
Most defense methods often sacrifice the accuracy of clean examples in order to
improve the adversarial robustness of DNNs. To alleviate these problems and
enhance the overall robust generalization of DNNs, we propose the Test-Time
Pixel-Level Adversarial Purification (TPAP) method. This approach is based on
the robust overfitting characteristic of DNNs to the fast gradient sign method
(FGSM) on training and test datasets. It utilizes FGSM for adversarial
purification, to process images for purifying unknown adversarial perturbations
from pixels at testing time in a "counter changes with changelessness" manner,
thereby enhancing the defense capability of DNNs against various unknown
adversarial attacks. Extensive experimental results show that our method can
effectively improve both overall robust generalization of DNNs, notably over
previous methods.