GRASP: A Rehearsal Policy for Efficient Online Continual Learning
Md Yousuf Harun Jhair Gallardo Christopher Kanan
Continual learning (CL) in deep neural networks (DNNs) involves incrementally
accumulating knowledge in a DNN from a growing data stream. A major challenge
in CL is that non-stationary data streams cause catastrophic forgetting of
previously learned abilities. Rehearsal is a popular and effective way to
mitigate this problem, which is storing past observations in a buffer and
mixing them with new observations during learning. This leads to a question:
Which stored samples should be selected for rehearsal? Choosing samples that
are best for learning, rather than simply selecting them at random, could lead
to significantly faster learning. For class incremental learning, prior work
has shown that a simple class balanced random selection policy outperforms more
sophisticated methods. Here, we revisit this question by exploring a new sample
selection policy called GRASP. GRASP selects the most prototypical (class
representative) samples first and then gradually selects less prototypical
(harder) examples to update the DNN. GRASP has little additional compute or
memory overhead compared to uniform selection, enabling it to scale to large
datasets. We evaluate GRASP and other policies by conducting CL experiments on
the large-scale ImageNet-1K and Places-LT image classification datasets. GRASP
outperforms all other rehearsal policies. Beyond vision, we also demonstrate
that GRASP is effective for CL on five text classification datasets.