In recent years, deep neural networks have demonstrated outstanding performancein many machine learning tasks. However, researchers have discovered that thesestate-of-the-art models are vulnerable to adversarial examples:  legitimate examples added by small perturbations which are unnoticeable to human eyes. Adversarial training, which augments the training data with adversarial examples duringthe training process,  is a well known defense to improve the robustness of themodel against adversarial attacks.   However, this robustness is only effective tothe same attack method used for adversarial training.   Madry et al. (2017) suggest that effectiveness of iterative multi-step adversarial attacks and particularlythat projected gradient descent (PGD) may be considered the universal first order adversary and applying the adversarial training with PGD implies resistanceagainst many other first order attacks.    However,  the computational cost of theadversarial training with PGD and other multi-step adversarial examples is muchhigher than that of the adversarial training with other simpler attack techniques. In this paper, we show how strong adversarial examples can be generated only ata cost similar to that of two runs of the fast gradient sign method (FGSM), allowing defense against adversarial attacks with a robustness level comparable to thatof the adversarial training with multi-step adversarial examples.   We empiricallydemonstrate the effectiveness of the proposed two-step defense approach againstdifferent attack methods and its improvements over existing defense strategies. Despite the fact that deep neural networks demonstrate outstanding performance for many machine learning tasks, researchers have found that they are susceptible to attacks by adversarial examples BID18 ; BID2 ). Adversarial examples which are generated by adding crafted perturbations to legitimate input samples are indistinguishable to human eyes. For classification tasks, these perturbations may cause the legitimate samples to be misclassified by the model at the inference time. While there exists no widely agreed conclusion, several studies attempted to explain the underlying causes of the susceptibility of deep neural networks toward adversarial examples. The vulnerability is ascribed to the linearity of the model BID2 ), low flexibility BID1 ), or the flatness/curvedness of the decision boundaries BID10 ), but a more general cause is still under research. The recent literature considered two types of threat models: black-box and white-box attacks. In black-box attacks, the attacker is assumed to have no access to the architecture and parameters of the model, whereas in white-box attacks, the attacker has complete access to such information. Several white-box attack methods were proposed BID2 , BID12 , BID17 , BID0 , BID9 ). In response, several defenses have been proposed to mitigate the effect of adversarial attacks. These defenses were developed along three main directions: (1) expanding the training data to make the classifier more robustly learn the underlying function, e.g., by adversarial training which augments the training data set with adversarial examples generated by certain attack methods BID18 , BID2 , BID5 ); (2) modifying the training procedure to reduce the gradients of the model w.r.t. the input such that the classifier becomes more robust to input perturbations, e.g., via input gradient regularization BID15 , or defensive distillation BID14 ; and (3) using external models as network add-ons when classifying unseen examples (feature squeezing BID19 , MagNet BID8 , and Defense-GAN) BID16 ).Adversarial training, a simple but effective method to improve the robustness of a deep neural network against white-box adversarial attacks, uses the same white-box attack mechanism to generate adversarial examples for augmenting the training data set. However, if the attacker applies a different attack strategy, adversarial training does not work well due to gradient masking BID13 . BID7 have suggested the effectiveness of iterative multi-step adversarial attacks. In particular, it was suggested that projected gradient descent (PGD) PGD may be considered the strongest first-order attack so that the adversarial training with PGD can boost the resistance against many other first-order attacks. However, in the literature a large number (e.g. 40) of steps of back propagation are typically used in the iterative attack method of PGD or its closely related variant iterative fast gradient (IFGSM) BID5 to find strong adversarial examples to be used in each adversarial training step, incurring a prohibitively high computational complexity particularly for large DNNs or training datasets.In this paper, we propose an efficient two-step adversarial defense technique, called e2SAD, to facilitate defense against multiple types of whitebox and blackbox attacks with a quality on a par with the expensive adversarial training using the well-known multi-step attack the iterative fast gradient method (IFGSM) BID5 . The first step of e2SAD is similar to the basic adversarial training, where an adversarial example is generated by applying a simple one-step attack method such as the fast gradient sign method (FGSM). Then in the second step, e2SAD attemps to generate a second adversarial example at which the vulnerability of the current model is maximally revealed such that the resulting defense is at the same quality level of the much more expensive IFGSMbased adversarial training. Finally, the two adversarial examples are taken into consideration in the proposed loss function according to which a more robust model is trained, resulting strong defense to both one-step and multi-step iterative attacks with a training time much less less than that of the adversarial training using IFGSM. The main contributions of this paper are as follows:• We propose a computationally efficient method to generate two adversarial examples per input example while effectively revealing the vulnerability of the learned classifier in the neighborhood of each clean data point;• We show that by considering the generated adversarial examples as part of a well-designed final loss function, the resulting model is robust to both one-step and iterative white box attacks;• We further demonstrate that by adopting other techniques in our two-step approach like the use of soft labels and hyper parameter tuning, robust defense against black box attacks can be achieved. We have aimed to improve the robustness of deep neural networks by presenting an efficient twostep adversarial defense technique e2SAD, particularly w.r.t to strong iterative multi-step attacks. This objective is achieved by finding a combination of two adversarial points to best reveal the vulnerability of the model around each clean input. In particular, we have demonstrated that using a dissimilarity measure between the first and second adversarial examples we are able to appropriately locate the second adversary in a way such that including both types of adversaries in the final training loss function leads to improved robustness against multi-step adversarial attacks. We have demonstrated the effectiveness of e2SAD in terms of defense against while-box one-step FGSM and multi-step IFGSM attacks and black-box IFGSM attacks under various settings.e2SAD provides a general mechanism for defending both one-step and multiple attacks and for balancing between these two defense needs, the latter of which can be achieved by properly tuning the corresponding weight hyperparameters in the training loss function. In the future work, we will explore hyperparameter tuning and other new techniques to provide a more balanced or further improved defense quality for a wider range of white and black box attacks.
We proposed a time-efficient defense method against one-step and iterative adversarial attacks. Propose a novel, computationaly efficient method named e2SAD which generates sets of two training adversarial samples for each clean training sample. The paper introduces a two-step adversarial defense method, to generate two adversarial examples per clean sample and include them in the actual training loop to achieve robustness and claiming it can outperform more expensive iterative methods. The paper presents a 2-step approach to generate strong adversarial examples at a far lesser cost as compared to recent iterative multi-step adversarial attacks.