We prove, under two sufficient conditions, that idealised models can have no adversarial examples. We discuss which idealised models satisfy our conditions, and show that idealised Bayesian neural networks (BNNs) satisfy these. We continue by studying near-idealised BNNs using HMC inference, demonstrating the theoretical ideas in practice. We experiment with HMC on synthetic data derived from MNIST for which we know the ground-truth image density, showing that near-perfect epistemic uncertainty correlates to density under image manifold, and that adversarial images lie off the manifold in our setting. This suggests why MC dropout, which can be seen as performing approximate inference, has been observed to be an effective defence against adversarial examples in practice; We highlight failure-cases of non-idealised BNNs relying on dropout, suggesting a new attack for dropout models and a new defence as well. Lastly, we demonstrate the defence on a cats-vs-dogs image classification task with a VGG13 variant. Adversarial examples, inputs to machine learning models that an adversary designs to manipulate model output, pose a major concern in machine learning applications. Many hypotheses have been suggested in the literature trying to explain the existence of adversarial examples. For example, BID31 hypothesise that these examples lie near the decision boundary, while BID22 hypothesise that these examples lie in low density regions of the input space. However, adversarial examples can lie far from the decision boundary (e.g. "garbage" images BID22 ), and using a simple spheres dataset it was shown that adversarial examples can exist in high density regions as well BID9 . In parallel work following BID22 's low-density hypothesis, BID17 empirically modelled input image density on MNIST and successfully detected adversarial examples by thresholding low input density. This puzzling observation, seemingly inconsistent with the spheres experiment in BID9 , suggests that perhaps additional conditions beyond the ability to detect low input density have led to the observed robustness by BID17 .Suggesting two sufficient conditions, here we prove that an idealised model (in a sense defined below) cannot have adversarial examples, neither in low density nor in high density regions of the input space. We concentrate on adversarial examples in discriminative classification models, models which are used in practical applications. To formalise our treatment, and to gain intuition into the results, we use tools such as discriminative Bayesian neural network (BNN) classifiers BID19 BID21 together with their connections to modern techniques in deep learning such as stochastic regularisation techniques BID6 . This pragmatic Bayesian perspective allows us to shed some new light on the phenomenon of adversarial examples. We further discuss which models other than BNNs abide by our conditions. Our hypothesis suggests why MC dropout-based techniques are sensible for adversarial examples identification, and why these have been observed to be consistently effective against a variety of attacks BID18 BID5 BID27 BID0 .We support our hypothesis mathematically and experimentally using HMC and dropout inference. We construct a synthetic dataset derived from MNIST for which we can calculate ground truth input densities, and use this dataset to demonstrate that model uncertainty correlates to input density, and that under our conditions this density is low for adversarial examples. Using our new-found insights we develop a new attack for MC dropout-based models which does not require gradient information, by looking for "holes" in the epistemic uncertainty estimation, i.e. imperfections in the uncertainty approximation, and suggest a mitigation technique as well. We give illustrative examples using MNIST BID15 , and experiment with real-world cats-vs-dogs image classification tasks BID3 ) using a VGG13 variant BID28 . Our result gives intuition into why dropout, a technique shown to relate to Bayesian modelling, seems to be effective in identifying adversarial examples. We presented several idealised models which satisfy the conditions we defined for robustness, opening the door for research into how various practical tools can approximate our idealised conditions. We highlighted that the main difficulty with modern BNNs is not coverage, but rather that approximate inference doesn't increase the un-certainty fast enough with practical BNN tools (we show this in figures 7a, demonstrating that we have holes in the dropout uncertainty). In contrast, HMC (which is not scalable for practical applications) does not have such uncertainty holes, suggesting that we must improve practical inference techniques in BNNs to improve robustness.
We prove that idealised Bayesian neural networks can have no adversarial examples, and give empirical evidence with real-world BNNs. The paper studies the adversarial robustness of Bayesian classifiers and state two conditions that they show are provably sufficient for "idealised models" on "idealised datasets" to not have adversarial examples Paper posit a class of discriminative BAyesian classifiers that do not have any adversarial examples.