Topology-Preserving Adversarial Training
Xiaoyue Mi Fan Tang Yepeng Weng Danding Wang Juan Cao Sheng Tang Peng Li Yang Liu
Despite the effectiveness in improving the robustness of neural networks,
adversarial training has suffered from the natural accuracy degradation
problem, i.e., accuracy on natural samples has reduced significantly. In this
study, we reveal that natural accuracy degradation is highly related to the
disruption of the natural sample topology in the representation space by
quantitative and qualitative experiments. Based on this observation, we propose
Topology-pReserving Adversarial traINing (TRAIN) to alleviate the problem by
preserving the topology structure of natural samples from a standard model
trained only on natural samples during adversarial training. As an additional
regularization, our method can easily be combined with various popular
adversarial training algorithms in a plug-and-play manner, taking advantage of
both sides. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet
show that our proposed method achieves consistent and significant improvements
over various strong baselines in most cases. Specifically, without additional
data, our proposed method achieves up to 8.78% improvement in natural accuracy
and 4.50% improvement in robust accuracy.