Deep neural networks have been shown to perform well in many classical machine learning problems, especially in image classification tasks. However, researchers have found that neural networks can be easily fooled, and they are surprisingly sensitive to small perturbations imperceptible to humans.   Carefully crafted input images (adversarial examples) can force a well-trained neural network to provide arbitrary outputs.   Including adversarial examples during training is a popular defense mechanism against adversarial attacks. In this paper we propose a new defensive mechanism under the generative adversarial network~(GAN) framework. We model the adversarial noise using a generative network, trained jointly with a classification discriminative network as a minimax game. We show empirically that our adversarial network approach works well against black box attacks, with performance on par with state-of-art methods such as ensemble adversarial training and adversarial training with projected gradient descent.
 Deep neural networks have been successfully applied to a variety of tasks, including image classification BID12 , speech recognition BID8 , and human-level playing of video games through deep reinforcement learning BID20 . However, BID29 showed that convolutional neural networks (CNN) are extremely sensitive to carefully crafted small perturbations added to the input images. Since then, many adversarial examples generating methods have been proposed, including Jacobian based saliency map attack (JSMA) BID23 , projected gradient descent (PGD) attack , and C&W's attack BID3 . In general, there are two types of attack models: white box attack and black box attack. Attackers in white box attack model have complete knowledge of the target network, including network's architecture and parameters. Whereas in black box attacks, attackers only have partial or no information on the target network BID25 .Various defensive methods have been proposed to mitigate the effect of the adversarial examples. Adversarial training which augments the training set with adversarial examples shows good defensive performance in terms of white box attacks BID13 . Apart from adversarial training, there are many other defensive approaches including defensive distillation BID24 , using randomization at inference time BID33 , and thermometer encoding BID2 , etc.In this paper, we propose a defensive method based on generative adversarial network (GAN) BID6 . Instead of using the generative network to generate samples that can fool the discriminative network as real data, we train the generative network to generate (additive) adversarial noise that can fool the discriminative network into misclassifying the input image. This allows flexible modeling of the adversarial noise by the generative network, which can take in the original image or a random vector or even the class label to create different types of noise. The discriminative networks used in our approach are just the usual neural networks designed for their specific classification tasks. The purpose of the discriminative network is to classify both clean and adversarial example with correct label, while the generative network aims to generate powerful perturbations to fool the discriminative network. This approach is simple and it directly uses the minimax game concept employed by GAN. Our main contributions include:• We show that our adversarial network approach can produce neural networks that are robust towards black box attacks. In the experiments they show similar, and in some cases better, performance when compared to state-of-art defense methods such as ensemble adversarial training BID30 and adversarial training with projected gradient descent . To our best knowledge we are also the first to study the joint training of a generative attack network and a discriminative network.• We study the effectiveness of different generative networks in attacking a trained discriminative network, and show that a variety of generative networks, including those taking in random noise or labels as inputs, can be effective in attacks. We also show that training against these generative networks can provide robustness against different attacks.The rest of the paper is organized as follows. In Section 2, related works including multiple attack and defense methods are discussed. Section 3 presents our defensive method in details. Experimental results are shown in Section 4, with conclusions of the paper in Section 5. In the experiments above we see that adversarial PGD training usually works best on white box attacks, but there is a tradeoff between accuracies on clean data against accuracies on adversarial examples due to finite model capacity. We can try to use models with larger capacity, but there is always a tradeoff between the two, especially for larger perturbations . There are some recent works that indicate training for standard accuracy and training for adversarial accuracy (e.g., with PGD) are two fairly different problems . Examples generated from PGD are particularly difficult to train against. This makes adversarial PGD training disadvantaged in many black box attack situations, when compared with models trained with weaker adversaries, e.g., ensemble adversarial training and our adversarial networks method.We have also observed in the experiments that for black box attacks, the most effective adversarial examples are usually those constructed from models trained using the same method but with different random seed. This suggests hiding the knowledge of the training method from the attacker could be an important factor in defending against black box attacks. Defending against black box attacks is closely related to the question of the transferability of adversarial examples. Although there are some previous works exploring this question BID15 , the underlying factors affecting transferability are still not well understood.In our experimentation with the architectures of the discriminative and generative networks, the choice of architectures of G φ does not seem to have a big effect on the quality of solution. The dynamics of training, such as the step size used and the number of iterations to run for each network during gradient descent/ascent, seem to have a bigger effect on the saddle point solution quality than the network architecture. It would be interesting to find classes of generative network architectures that lead to substantially different saddle points when trained against a particular discriminative network architecture. Also, recent works have shown that there are connected flat regions in the minima of neural network loss landscapes BID5 BID4 . We believe that the same might hold true for GANs, and it would be interesting to explore how the training dynamics can lead to different GAN solutions that might have different robustness properties.Our approach can be extended with multiple discriminative networks playing against multiple generative networks. It can also be combined with ensemble adversarial training, where some adversarial examples come from static pre-trained models, while some other come from dynamically adjusting generative networks. We have proposed an adversarial network approach to learning discriminative neural networks that are robust to adversarial noise, especially under black box attacks. For future work we are interested in extending the experiments to ImageNet, and exploring the choice of architectures of the discriminative and generative networks and their interaction. generative networks used in this paper. G0 and G1 are encoder-decoder networks, while G2 and G3 are decoder networks using a random vector and a one-hot encoding of the label respectively. The generative networks are parameterized by a factor k determining the number of filters used (width of network). As default we use k = 64, and k = 16 for networks using labels as inputs.EXTRA RESULTS ON CIFAR100 AND WIDE RESNET ON CIFAR10The discriminative and generative networks in our CIFAR100 experiment have the same network architecture as the CIFAR10 experiment, except that the output layer dimension of the D network is 100 other than 10 in CIFAR10. We use learning rate of 0.1 for the first 100k iterations, and 0.01 for another 100k iterations. The batch size is 64 and weight decay is 1E-5. TAB8 gives the results on CIFAR10 using a wider version of Resnet (Model D2), by multiplying the number of filters in each convolutional layer by a factor of 10. Some of the previous works in the literature use models of larger capacity for training adversarially robust models, so we perform experiments on these large capacity models here. First the accuracies increase across the board with larger capacity models. The accuracy gap on clean data between adversarial PGD and standard training still exists, but now there is also a small accuracy gap between our adversarial network approach and standard training. For the rest of the white box and black accuracies the story is similar, the models are weakest against attacks trained with the same method but with a different random seed. Our adversarial network approach has very good performance across different attacks, even as it is not always the winner for each individual attack. TAB10 gives the results of Wide ResNet on CIFAR100, and the results are qualitatively similar.
Jointly train an adversarial noise generating network with a classification network to provide better robustness to adversarial attacks. A GAN solution for deep models of classification, faced to white and black box attacks, that produces robust models.  The paper proposes a defensive mechanism against adversarial attacks using GANs with generated perturbations used as adversarial examples and a discriminator used to distinguish between them