Deep neural networks (DNNs) are vulnerable to adversarial examples, which are carefully crafted instances aiming to cause prediction errors for DNNs. Recent research on adversarial examples has examined local neighborhoods in the input space of DNN models. However, previous work has limited what regions to consider, focusing either on low-dimensional subspaces or small balls. In this paper, we argue that information from larger neighborhoods, such as from more directions and from greater distances, will better characterize the relationship between adversarial examples and the DNN models. First, we introduce an attack, OPTMARGIN, which generates adversarial examples robust to small perturbations. These examples successfully evade a defense that only considers a small ball around an input instance. Second, we analyze a larger neighborhood around input instances by looking at properties of surrounding decision boundaries, namely the distances to the boundaries and the adjacent classes. We find that the boundaries around these adversarial examples do not resemble the boundaries around benign examples. Finally, we show that, under scrutiny of the surrounding decision boundaries, our OPTMARGIN examples do not convincingly mimic benign examples. Although our experiments are limited to a few specific attacks, we hope these findings will motivate new, more evasive attacks and ultimately, effective defenses. Recent research in adversarial examples in deep learning has examined local neighborhoods in the input space of deep learning models. BID9 and BID15 examine limited regions around benign samples to study why some adversarial examples transfer across different models. BID10 explore regions around benign samples to validate the robustness of an adversarially trained model. BID14 examine regions around adversarial examples to estimate the examples' robustness to random noise. BID0 determine that considering the region around an input instance produces more robust classification than looking at the input instance alone as a single point. In this paper, we argue that information from larger neighborhoods-both in more directions and at greater distances-will better help us understand adversarial examples in high-dimensional datasets.First, we describe a concrete limitation in a system that utilizes information in small neighborhoods. Cao & Gong's region classification defense (2017) takes the majority prediction in a small ball around an input instance. We introduce an attack method, OPTMARGIN, for generating adversarial examples that are robust to small perturbations, which can evade this defense.Second, we provide an example of how to analyze an input instance's surroundings in the model's input space. We introduce a technique that looks at the decision boundaries around an input instance, and we use this technique to characterize our robust OPTMARGIN adversarial examples. Our analysis reveals that, while OPTMARGIN adversarial examples are robust enough to fool region classification, the decision boundaries around them do not resemble the boundaries around benign examples, in terms of distances from the example to the adjacent classes.Third, as an extension to the above observation, we train a classifier to differentiate the decision boundary information that comes from different types of input instances. We show that our classifier can differentiate OPTMARGIN and benign examples with 90.4% accuracy, whereas region classification limits itself to a small region and fails. However, it remains to be seen whether a more sophisticated attack can find adversarial examples surrounded by decision boundaries that more accurately mimic the boundaries around benign examples.To summarize, our contributions are:1. We demonstrate OPTMARGIN, a new attack that evades region classification systems with low-distortion adversarial examples.2. We introduce an analysis of decision boundaries around an input instance that explains the effectiveness of OPTMARGIN adversarial examples and also shows the attack's weaknesses.3. We demonstrate the expressiveness of decision boundary information by using it to classify different kinds of input instances.We have released the code we used at https://github.com/sunblaze-ucb/ decision-boundaries. We considered the benefits of examining large neighborhoods around a given input in input space. We demonstrated an effective OPTMARGIN attack against a region classification defense, which only considered a small ball of the input space around a given instance. We analyzed the neighborhood of examples generated by this new attack by looking at the decision boundaries around them, as well as the boundaries around benign examples and less robust adversarial examples. This analysis incorporated information from many directions in input space and from longer distances than previous work. We found that the comprehensive information about surrounding decision boundaries reveals there are still differences between our robust adversarial examples and benign examples.
Looking at decision boundaries around an input gives you more information than a fixed small neighborhood The authors present a novel attack for generating adversarial examples where they attack classifiers created by randomly classifying L2 small perturbations A new approach to generate adversarial attacks to a neural network, and a method to defend a neural network from those attacks.