Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks as they train on single classification tasks. Our goal is to understand whether a related phenomenon occurs when data does not undergo a clear distributional shift. We define a ``forgetting event'' to have occurred when an individual training example transitions from being classified correctly to incorrectly over the course of learning. Across several benchmark data sets, we find that: (i) certain examples are forgotten with high frequency, and some not at all; (ii) a data set's (un)forgettable examples generalize across neural architectures; and (iii) based on forgetting dynamics, a significant fraction of examples can be omitted from the training data set while still maintaining state-of-the-art generalization performance. Many machine learning models, in particular neural networks, cannot perform continual learning. They have a tendency to forget previously learnt information when trained on new tasks, a phenomenon usually called catastrophic forgetting BID17 BID29 . One of the hypothesized causes of catastrophic forgetting in neural networks is the shift in the input distribution across different tasks-e.g., a lack of common factors or structure in the inputs of different tasks might lead standard optimization techniques to converge to radically different solutions each time a new task is presented. In this paper, we draw inspiration from this phenomenon and investigate the extent to which a related forgetting process occurs as a model learns examples traditionally considered to belong to the same task.Similarly to the continual learning setting, in stochastic gradient descent (SGD) optimization, each mini-batch can be considered as a mini-"task" presented to the network sequentially. In this context, we are interested in characterizing the learning dynamics of neural networks by analyzing (catastrophic) example forgetting events. These occur when examples that have been "learnt" (i.e., correctly classified) at some time t in the optimization process are subsequently misclassifiedor in other terms forgotten -at a time t > t. We thus switch the focus from studying interactions between sequentially presented tasks to studying interactions between sequentially presented dataset examples during SGD optimization. Our starting point is to understand whether there exist examples that are consistently forgotten across subsequent training presentations and, conversely, examples that are never forgotten. We will call the latter unforgettable examples. We hypothesize that specific examples consistently forgotten between subsequent presentations, if they exist, must not share commonalities with other examples from the same task. We therefore analyze the proportion of forgettable/unforgettable examples for a given task and what effects these examples have on a model's decision boundary and generalization error.The goal of our investigation is two-fold. First, we attempt to gain insight into the optimization process by analyzing interactions among examples during learning and their influence on the final decision boundary. We are particularly interested in whether we can glean insight on the compressibility of a dataset, and thereby increase data efficiency without compromising generalization accuracy. It is a timely problem that has been the recent focus of few-shot learning approaches via meta-learning BID8 BID28 . Second, we aim to characterize whether forgetting statistics can be used to identify "important" samples and detect outliers and examples with noisy labels BID12 BID3 BID32 BID11 .Identifying important, or most informative examples is an important line of work and was extensively studied in the literature. Techniques of note -among others -are predefined curricula of examples BID1 , self-paced learning BID21 , and more recently meta-learning BID7 . These research directions usually define "hardness" or "commonality" of an example as a function of the loss on that particular example at some point during training (or possibly at convergence). They do not consider whether some examples are consistently forgotten throughout learning. Very recently, BID4 consider re-weighting examples by accounting for the variance of their predictive distribution. This is related to our definition of forgetting events, but the authors provide little analysis of the extent to which the phenomenon occurs in their proposed tasks. Our purpose is to study this phenomenon from an empirical standpoint and characterize its prevalence in different datasets and across different model architectures.Our experimental findings suggest that: a) there exist a large number of unforgettable examples, i.e., examples that are never forgotten once learnt, those examples are stable across seeds and strongly correlated from one neural architecture to another; b) examples with noisy labels are among the most forgotten examples, along with images with "uncommon" features, visually complicated to classify; c) training a neural network on a dataset where a very large fraction of the least forgotten examples have been removed still results in extremely competitive performance on the test set. In this paper, inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks when training on single classification tasks. We show that catastrophic forgetting can occur in the context of what is usually considered to be a single task. Inspired by this result, we find that some examples within a task are more prone to being forgotten, while others are consistently unforgettable. We also find that forgetting statistics seem to be fairly stable with respect to the various characteristics of training, suggesting that they actually uncover intrinsic properties of the data rather than idiosyncrasies of the training schemes. Furthermore, the unforgettable examples seem to play little part in the final performance of the classifier as they can be removed from the training set without hurting generalization. This supports recent research interpreting deep neural networks as max margin classifiers in the linear case. Future work involves understanding forgetting events better from a theoretical perspective, exploring potential applications to other areas of supervised learning, such as speech or text and to reinforcement learning where forgetting is prevalent due to the continual shift of the underlying distribution. permutedMNIST The permutedMNIST data set is obtained by applying a fixed random permutation of the pixels to all the images of the standard MNIST data set. This typically makes the data set harder to learn for convolutional neural networks as local patterns, e.g. the horizontal bar of the 7, get shuffled. This statement is supported by the two following facts:
We show that catastrophic forgetting occurs within what is considered to be a single task and find that examples that are not prone to forgetting can be removed from the training set without loss of generalization. Studies the forgetting behavior of training examples during SGD, and shows there exist "support examples" in neural network training across different network architectures. This paper analyzes the extent to which networks learn to correctly classify specific examples and then forget these examples over the course of training. The paper studies whether some examples in training neural networks are harder to learn than others. Such examples are forgotten and relearned multiple times through learning.