Current machine learning algorithms can be easily fooled by adversarial examples. One possible solution path is to make models that use confidence thresholding to avoid making mistakes. Such models refuse to make a prediction when they are not confident of their answer. We propose to evaluate such models in terms of tradeoff curves with the goal of high success rate on clean examples and low failure rate on adversarial examples. Existing untargeted attacks developed for models that do not use confidence thresholding tend to underestimate such models' vulnerability. We propose the MaxConfidence family of attacks, which are optimal in a variety of theoretical settings, including one realistic setting: attacks against linear models. Experiments show the attack attains good results in practice. We show that simple defenses are able to perform well on MNIST but not on CIFAR, contributing further to previous calls that MNIST should be retired as a benchmarking dataset for adversarial robustness research.   We release code for these evaluations as part of the cleverhans (Papernot et al 2018) library  (ICLR reviewers should be careful not to look at who contributed these features to cleverhans to avoid de-anonymizing this submission). We have made the following contributions:• We have shown that adversarial training on one kind of out-of-distribution data can actually worsen performance on other kinds of out-of-distribution data, relative to a baseline that uses confidence thresholding as the only defense.• We have introduced the evaluation methodology of success-fail curves, showing which success rates on clean data and failure rates on adversarial data are feasible for different confidence thresholds.• We have presented an attack that is optimal against a variety of confidence thresholding models. Specifically , it is optimal against linear classifiers and optimal against general models whenever the underlying optimization approximately succeeds.• We have shown an evaluation methodology that maps out an entire success-failure tradeoff curve without needing to re-train the model or re-run the evaluation for different thresholds.• We have shown that confidence thresholding with simple regularization is sufficient to achieve reasonable robustness to L ∞ attacks on MNIST, despite being roughly 40X cheaper to train than adversarial training.• We have shown that confidence thresholding can lead to robustness to a variety of attacks, without needing to anticipate and formally specify each attack type.Overall, we hope that our evaluation methodology will help to design and rigorously test low-cost, versatile defenses against a wide variety of adversarial examples. A set of points that should be equivalent to x. An adversarial example corresponding to x must be drawn from this set. x An adversarial example corresponding to x p model (y | x) The conditional distribution over the classes represented by the model c(x)arg max y p model (y | x), the confidence of the model for input x kThe number of classes tThe confidence threshold used by the model. An input x is covered only if c(x) > t. wThe weight vector for a logistic regression model η A perturbation applied to a clean input x B MODEL A Our "Model A" is a simple model that we tuned by trial and error to yield better success-fail curves than the baseline. We do not advocate "Model A" as the latest and greatest model that everyone should switch to. It is only included as a test point to show that our evaluation methodology can find interesting differences between models that have similar accuracy at 100% coverage on clean and adversarial data.Our trial-and-error design process was based on performance on clean data and on L ∞ adversarial examples. We did not use information about performance on semantic adversarial examples during the design process, so the defense was not designed in any specific way to handle these examples.The model architecture is straightforward to describe in cleverhans format: Conv2D(nb_filters, (3, 3) , (2, 2), "SAME"), ReLU(), Add ([Conv2D(nb_filters, (3, 3) , (1, 1), "SAME"), ReLU(), Conv2D(nb_filters, (3, 3) , (1, 1), "SAME")]), Conv2D(nb_filters * 2, (3, 3), (2, 2), "SAME"), ReLU(), Conv2D(nb_filters * 2, (3, 3), (1, 1), "VALID"), ReLU(), Flatten(), Linear(nb_classes), Softmax()] DISPLAYFORM0 In other words, it is a simple convolutional network, containing a convolution and 2X downsampling layer, a residual layer, two convolutional layers, and a fully connected layer to output the logits. There are no normalization layers, etc., and all of the hidden units are ReLUs BID9 BID14 BID3 .
We present metrics and an optimal attack for evaluating models that defend against adversarial examples using confidence thresholding This paper introduces a family of attack on confidence thresholding algortihms, focusing mainly on evaluation methodologies. Proposes an evaluation method for confidence thresholding defense models and an approach for generating adversarial examples by choosing the wrong class with the most confidence when using targeted attacks The paper presents an evaluation methodology for evaluating attacks on confidence thresholding methods and proposes a new kind of attack.