Fast Adversarial Training against Textual Adversarial Attacks
Yichen Yang Xin Liu Kun He
Many adversarial defense methods have been proposed to enhance the
adversarial robustness of natural language processing models. However, most of
them introduce additional pre-set linguistic knowledge and assume that the
synonym candidates used by attackers are accessible, which is an ideal
assumption. We delve into adversarial training in the embedding space and
propose a Fast Adversarial Training (FAT) method to improve the model
robustness in the synonym-unaware scenario from the perspective of single-step
perturbation generation and perturbation initialization. Based on the
observation that the adversarial perturbations crafted by single-step and
multi-step gradient ascent are similar, FAT uses single-step gradient ascent to
craft adversarial examples in the embedding space to expedite the training
process. Based on the observation that the perturbations generated on the
identical training sample in successive epochs are similar, FAT fully utilizes
historical information when initializing the perturbation. Extensive
experiments demonstrate that FAT significantly boosts the robustness of BERT
models in the synonym-unaware scenario, and outperforms the defense baselines
under various attacks with character-level and word-level modifications.