It is well known that it is possible to construct "adversarial examples"
 for neural networks: inputs which are misclassified by the network
 yet indistinguishable from true data. We propose a simple
 modification to standard neural network architectures, thermometer
 encoding, which significantly increases the robustness of the network to
 adversarial examples. We demonstrate this robustness with experiments
 on the MNIST, CIFAR-10, CIFAR-100, and SVHN datasets, and show that
 models with thermometer-encoded inputs consistently have higher accuracy
 on adversarial examples, without decreasing generalization.
 State-of-the-art accuracy under the strongest known white-box attack was 
 increased from 93.20% to 94.30% on MNIST and 50.00% to 79.16% on CIFAR-10.
 We explore the properties of these networks, providing evidence
 that thermometer encodings help neural networks to
 find more-non-linear decision boundaries. Adversarial examples are inputs to machine learning models that are intentionally designed to cause the model to produce an incorrect output. The term was introduced by Szegedy et al. (2014) in the context of neural networks for computer vision. In the context of spam and malware detection, such inputs have been studied earlier under the name evasion attacks BID0 . Adversarial examples are interesting from a scientific perspective, because they demonstrate that even machine learning models that have superhuman performance on I.I.D. test sets fail catastrophically on inputs that are modified even slightly by an adversary. Adversarial examples also raise concerns in the emerging field of machine learning security because malicious attackers could use adversarial examples to cause undesired behavior BID15 .Unfortunately , there is not yet any known strong defense against adversarial examples. Adversarial examples that fool one model often fool another model, even if the two models are trained on different training examples (corresponding to the same task) or have different architectures (Szegedy et al., 2014) , so an attacker can fool a model without access to it. Attackers can improve their success rate by sending inputs to a model, observing its output, and fitting their own own copy of the model to the observed input-output pairs BID15 . Attackers can also improve their success rate by searching for adversarial examples that fool multiple different models-such adversarial examples are then much more likely to fool the unknown target model . Szegedy et al. (2014) proposed to defend the model using adversarial training (training on adversarial examples as well as regular examples) but it was not feasible to generate enough adversarial examples in the inner loop of the training process for the method to be effective at the time. Szegedy et al. (2014) used a large number of iterations of L-BFGS to produce their adversarial examples. BID3 developed the fast gradient sign method (FGSM) of generating adversarial examples and demonstrated that adversarial training is effective for reducing the error rate on adversarial examples. A major difficulty of adversarial training is that it tends to overfit to the method of adversarial example generation used at training time. For example, models trained to resist FGSM adversarial examples usually fail to resist L-BFGS adversarial examples. BID9 introduced the basic iterative method (BIM) which lies between FGSM and L-BFGS on a curve trading speed for effectiveness (the BIM consists of running FGSM for a medium number of iterations). Adversarial training using BIM still overfits to the BIM, unfortunately, and different iterative methods can still successfully attack the model. Recently, BID13 showed that adversarial training using adversarial examples created by adding random noise before running BIM results in a model that is highly robust against all known attacks on the MNIST dataset. However, it is less effective on more complex datasets, such as CIFAR. A strategy for training networks which are robust to adversarial attacks across all contexts is still unknown. In this work, we demonstrate that thermometer code discretization and one-hot code discretization of real-valued inputs to a model significantly improves its robustness to adversarial attack, advancing the state of the art in this field. In BID3 , the seeming linearity of deep neural networks was shown by visualizing the networks in several different ways. To test our hypothesis that discretization breaks some of this linearity, we replicate these visualizations and contrast them to visualizations of discretized models. See Appendix G for an illustration of these properties.For non-discretized, clean trained models, test-set examples always yield a linear boundary between correct and incorrect classification; in contrast, non-adversarially-trained models have a more interesting parabolic shape (see Figure 9 ). Loss for iterated white-box attacks on various models on a randomly chosen data point from MNIST. By step 40, which is where we evaluate, the loss of the point found by iterative attacks has converged. DISPLAYFORM0 When discretizing the input, we introduce C w · C h · C o · c · (k − 1) extra parameters, where c is the number of channels in the image, k is the number of levels of discretization, and C w , C h , C o are the width, height, and output channels of the first convolutional layer. Discretizing using 16 levels introduced 0.03% extra parameters for MNIST, 0.08% for CIFAR-10 and CIFAR-100, and 2.3% for SVHN. This increase is negligible, so it is likely that the robustness comes from the input discretization, and is not merely a byproduct of having a slightly higher-capacity model.
Input discretization leads to robustness against adversarial examples The authors present an in-depth study of discretizing / quantizing the input as a defense against adversarial examples