Convolutional Neural Networks (CNNs) significantly improve the state-of-the-art for many applications, especially in computer vision. However, CNNs still suffer from a tendency to confidently classify out-distribution samples from unknown classes into pre-defined known classes. Further, they are also vulnerable to adversarial examples. We are relating these two issues through the tendency of CNNs to over-generalize for areas of the input space not covered well by the training set. We show that a CNN augmented with an extra output class can act as a simple yet effective end-to-end model for controlling over-generalization. As an appropriate training set for the extra class, we introduce two resources that are computationally efficient to obtain: a representative natural out-distribution set and interpolated in-distribution samples. To help select a representative natural out-distribution set among available ones, we propose a simple measurement to assess an out-distribution set's fitness. We also demonstrate that training such an augmented CNN with representative out-distribution natural datasets and some interpolated samples allows it to better handle a wide range of unseen out-distribution samples and black-box adversarial examples without training it on any adversaries. Finally, we show that generation of white-box adversarial attacks using our proposed augmented CNN can become harder, as the attack algorithms have to get around the rejection regions when generating actual adversaries. Convolutional Neural Networks (CNNs) have allowed for significant improvements over the stateof-the-art in the last few years for various applications, and in particular for computer vision. Notwithstanding these successes, challenging issues remain with these models. In the following work, we specifically look at two concerns. First, CNNs are vulnerable to different types of adversarial examples BID26 BID13 BID3 . These adversarial examples are created by deliberately modifying clean samples with imperceptible perturbations, with the aim of misleading CNNs into classifying them to a wrong class with high confidence. Second, CNNs are not able to handle instances coming from outside the task domain on which they are trained -the so-called out-distribution samples BID18 BID15 . In other words, although these examples are semantically and statistically different from the (in-distribution) samples relevant to a given task, the neural network trained on the task assigns such out-of-concept samples with high-confidence to the pre-defined in-distribution classes. Due to the susceptibility of CNNs to both adversaries and out-distribution samples, deploying them for real-world applications, in particular for security-sensitive ones, is a serious concern. These two issues have been treated separately in the past, with two distinct family of approaches. For instance, on the one hand, to handle out-distribution samples, some researchers have proposed threshold-based post-processing approaches with the aim of firstly calibrating the predictive confidence scores provided by either a single pre-trained CNN BID18 BID10 BID17 or an ensemble of CNNs BID15 , and then detecting out-distribution samples according to an optimal threshold. However, it is difficult to define an optimal and stable threshold for rejecting a wide range of out-distribution samples without increasing the false negative rate (i.e., rejecting in-distribution samples). On the other hand, researchers regarded adversarial examples as a distinct issue from the out-distribution problem and attempted to either correctly classify all adversaries through adversarial training of CNNs BID27 BID7 or reject all of them by training a separate detector BID5 BID20 . The performance of these approaches at properly handling adversarial instances mostly depends on having access to a diverse set of training adversaries, which is not only computationally expensive but also handling some possible future adversaries, which have not been discovered yet, most likely is difficult.It is known that deep neural networks (e.g. CNNs) are prone to over-generalization in the input space by partitioning it entirely into a set of pre-defined classes for a given in-distribution set (task), regardless of the fact that in-distribution samples may only be relevant to a small portion of the input space BID18 BID25 BID0 . In this paper, we highlight that the two aforementioned issues of CNNs can be alleviated simultaneously through control of over-generalization. To this end, we propose that an augmented CNN, a regular (naive) CNN with an extra class dubbed as dustbin, can be a simple yet effective solution, if it is trained on appropriate training samples for the dustbin class. Furthermore, we introduce here a computationally-efficient answer to the following key question: how to acquire such an appropriate set to effectively reduced the over-generalized regions induced by naive CNN. We note that our motivation for employing an augmented CNN is different from the threshold-based post-processing approaches that attempt to calibrate the predictive confidence scores of a pre-trained naive CNN without impacting its feature space. Our motivation in fact is to learn a more expressive feature space, where along with learning the sub-manifolds corresponding to in-distribution classes, a distinct extra sub-manifold for the dustbin class can be obtained such that the samples drawn from many over-generalized regions including a wide-range of out-distribution samples and various types of adversaries are mapped to this "dustbin" sub-manifold.As a training source for the extra class (dustbin), one can consider using synthetically generated out-distribution samples BID17 BID11 or adversarial examples BID8 . However, using such generated samples is not only computationally expensive but also barely able to effectively reduce over-generalization compared to naive CNNs (see Sec. 3). Instead of such synthetic samples, there are plenty of cost-effective training sources available for the extra dustbin class, namely natural out-distribution datasets. By natural out-distribution sets we mean the sets containing some realistic (not synthetically generated) samples that are semantically and statistically different from those in the in-distribution set. A representative natural out-distribution set for a given in-distribution task should be able to adequately cover the over-generalized regions. To recognize such a representative natural set, we propose a simple measurement to assess its fitness for a given in-distribution set. In addition to the selected set, we generate some artificial out-distribution samples through a straightforward and computationally efficient procedure, namely by interpolating some pair of in-distribution samples. We believe a properly trained augmented CNN can be utilized as a threshold-free baseline for identifying concurrently a broad range of unseen out-distribution samples and different types of strong adversarial attacks.The main contributions of the paper are summarized as:• By limiting the over-generalization regions induced by naive CNNs, we are able to drastically reduce the risk of misclassifying both adversaries and samples from a broad range of (unseen) out-distribution sets. To this end, we demonstrate that an augmented CNN can act as a simple yet effective solution.• We introduce a measurement to select a representative natural out-distribution set among those available for training effective augmented CNNs, instead of synthesizing some dustbin samples using hard-to-train generators.• Based on extensive experiments on a range of different image classification tasks, we demonstrate that properly trained augmented CNNs can significantly reduce the misclassification rates for both 1 ) unseen out-distribution sets, and 2) for various types of strong black-box adversarial examples, even though they are never trained on any specific types of adversaries.• For the generation of white-box adversaries using our proposed augmented CNN, the adversarial attack algorithms frequently encounter dustbin regions rather than regions from other classes when distorting a clean samples, making the adversaries generation process more difficult. In this paper we bridge two issues of CNNs that were previously thought of as unrelated: susceptibility of naive CNNs to various types of adversarial examples and incorrect high confidence prediction for out-distribution samples. We argue these two issues are connected through over-generalization. We propose augmented CNNs as a simple yet effective solution for controlling over-generalization, when they are trained on an appropriate set of dustbin samples. Through empirical evidence, we define an indicator for selecting an "appropriate" natural out-distribution set as training samples for dustbin class from among those available and show such selection plays a vital role for training effective augmented CNNs. Through extensive experiments on several augmented CNNs in different settings, we demonstrate that reducing over-generalization can significantly reduce the misclassification error rates of CNNs on adversaries and out-distribution samples, simultaneously, while their accuracy rates on in-distribution samples are maintained. Indeed, reducing over-generalization by such an end-to-end learning model (e.g., augmented CNNs) leads to learning more expressive feature space where these two categories of hostile samples (i.e., adversaries and out-distribution samples) are disentangled from in-distribution samples.
Properly training CNNs with dustbin class increase their robustness to adversarial attacks and their capacity to deal with out-distribution samples. This paper proposes adding an additional label for detecting OOD samples and adversarial examples in CNN models. The paper proposes an additional class that incorporates natural out-distribution images and interpolated images for adversarial and out-distribution samples in CNNs