We introduce the deep abstaining classifier -- a deep neural network trained with a novel loss function that provides an abstention option during training. This allows the  DNN to abstain on confusing or difficult-to-learn examples while improving performance on the non-abstained samples. We show that such deep abstaining classifiers can: (i) learn representations for structured noise -- where noisy training labels or confusing examples are correlated with underlying features -- and then learn to abstain based on such features; (ii) enable robust learning in the presence of arbitrary or unstructured noise by identifying noisy samples; and (iii) be used as an effective out-of-category detector that learns to reliably abstain when presented with samples from  unknown classes. We provide analytical results on loss function behavior that enable automatic tuning of accuracy and coverage, and demonstrate the utility of the deep abstaining classifier using multiple image benchmarks, Results indicate significant improvement in learning in the presence of label noise. Machine learning algorithms are expected to increasingly replace humans in decision-making pipelines. With the deployment of AI-based systems in high risk fields such as medical diagnosis BID23 , autonomous vehicle control BID19 and the legal sector BID3 , an erroneous prediction that should have otherwise been flagged for human interventionbecause the system has not robustly learned when it is likely to get the wrong answer -can have severe consequences.In these situations, the quality of "knowing when it doesn't know" and abstaining from predicting is an essential trait for a classifier to possess. This allows the decision-making to be routed to a human or another more accurate, but possibly more expensive, classifier, with the assumption being that the additional cost incurred is greatly surpassed by the consequences of a wrong prediction.Since learning systems have been around for multiple decades, there has been extensive theoretical and empirical investigation into rejection (or abstention) classification with the bulk of this being in the area of shallow learners BID5 BID7 BID10 and multilayer perceptrons BID8 . A framework for "self-aware learning" was analyzed in the context of Markov decision processes in BID20 . In the context of deep networks, this has been an under-explored area with BID11 recently proposing an effective technique of selective classification for optimizing risk-vs-coverage profiles based on the output of a trained model.The abstention formulations in all the previous works have been in a post-processing setting i.e., a classifier is first trained as usual, and an abstention threshold is determined based on post-training performance on a calibration set. In this paper, we introduce a method to train DNNs that utilizes an abstention option while training. Using a novel loss function -a modified version of the multi-class categorical cross-entropy loss that includes an abstention output -the representational capacity of a DNN is exploited for learning when abstention is a better option, while at the same improving performance on the non-abstained samples. We illustrate the utility of a DNN trained in this way in multiple situations: first, when labeling errors are correlated with some underlying feature of the data (systematic or structured label noise), abstention training allows the DNN to learn features that are indicative of unreliable training signals and which are thus likely to lead to uncertain predictions. This kind of representation learning for abstention is useful both for effectively eliminating structured noise and also for interpreting the reasons for abstention. Second, we show how an abstention-based approach can be used as a very effective data cleaner when training data contains arbitrary (or unstructured) label noise: a DNN trained with an abstention option can be used to identify and filter out noisy training data leading to significant performance benefits for downstream training using a cleaner set. Finally, we also consider the problem of open-set detection since real-world systems are often deployed in open-domain situations; when presented with samples from unknown classes, abstention is often the safest choice. We describe a method for utilizing abstention training for effective open-set detection by training the DNN to pickup only features associated with known classes, and abstain when such features are absent. To summarize, the contributions of this paper are:• The introduction of the deep abstaining classifier (DAC) -a DNN trained with a novel loss function that uses an abstention class while training -enabling robust learning in the presence of label noise.• The demonstration of the ability of the DAC to learn features associated with systematic label noise. Through numerous experiments, we show how the DAC is able to pick up (and then abstain on) such features with remarkable precision.• Demonstration of the utility of the DAC as a highly effective data cleaner in the presence of arbitrary label noise. We provide results on learning with noisy labels on multiple image benchmarks (CIFAR-10, CIFAR-100 and Fashion-MNIST) that are competitive and in many cases, significantly improve performance compared to existing methods.• Illustration of the DAC as an effective open-set detector that learns to reliably abstain when presented with samples from unknown classes.We note that, while ideally such an abstaining classifier should also learn to reliably abstain when presented with adversarially perturbed samples BID26 BID35 MoosaviDezfooli et al., 2017) , in this work we do not consider adversarial settings and leave that for future exploration. The rest of the paper is organized as follows: Section 2 describes the loss function formulation and an algorithm for automatically tuning abstention behavior. Section 3 discusses learning in the presence of structured noise, including experimental results and visual interpretations of abstention. Section 4 presents the utility of the DAC for data cleaning in the presence of unstructured (arbitrary) noise. Section 5 has further discussions on abstention behavior in the context of memorization. Section 6 discusses open set detection with the DAC. We conclude in Section 7. We introduced and the illustrated the utility of a deep abstaining classifier -a DNN trained on a novel loss function that learns to abstain as opposed to abstention calibration after training. We illustrated the utility of the DAC in multiple situations: as a representation learner in the presence of structured noise, as an effective data cleaner in the presence of arbitrary noise, and as an effective out-of-category detector. While adversarial settings were not considered in this work, the DAC, and abstention in general, might be considered as part of the defense arsenal against adversarial attacks; we leave this for future work. In summary, our results here indicate that the representational power of DNNs can be used very effectively as a means of self-calibration -"knowing when it doesn't know". Figure 6 : Results on blurred-image experiment with noisy labels (a)20% of the images are blurred in the train set, and their labels randomized (b) Validation accuracy for baseline vs DAC (non-abstained) (c)Abstention behavior for the DAC during training (d) Distribution of predictions on the blurred validation images for the DAC. We also observed (not shown) that for the baseline DNN, the accuracy on the blurred images in the validation set is no better than random.Results The DAC abstains remarkably well on the blurred images in the test set (Figure 6d ), while maintaining classification accuracy over the remaining samples in the validation set (≈ 79%). The baseline DNN accuracy drops to 63% (Figure 6b ), while the basline accuracy over the smudged images alone is no better than random (≈ 9.8%) . The abstention behavior of the DAC on the blurred images in the test set can be explained by how abstention evolves during training (Figure 6c ). Once abstention is introduced at epoch 20, the DAC initially opts to abstain on a high percentage of the traning data, while continuing to learn (since the gradients w.r.t the true-class pre-activations are always negative.). In the later epochs, sufficient learning has taken place on the non-randomized samples but the DAC continues to abstain on about 20% of the training data, which corresponds to the blurred images indicating that a strong association has been made between blurring and abstention.
A deep abstaining neural network trained with a novel loss function that learns representations for when to abstain enabling robust learning in the presence of different types of noise. A new loss function for training a deep neural network which can abstain, with performance looked at from angles in existence of structured noise, in existence of unstructured noise, and open world detection. This manuscript introduces deep abstaining classifiers which modifies the multiclass cross-entropy loss with an abstention loss, which is then applied to perturbed image classification tasks