Out-of-distribution forgetting: vulnerability of continual learning to
  intra-class distribution shift
Liangxuan Guo Yang Chen Shan Yu
Continual learning (CL) is an important technique to allow artificial neural
networks to work in open environments. CL enables a system to learn new tasks
without severe interference to its performance on old tasks, i.e., overcome the
problems of catastrophic forgetting. In joint learning, it is well known that
the out-of-distribution (OOD) problem caused by intentional attacks or
environmental perturbations will severely impair the ability of networks to
generalize. In this work, we reported a special form of catastrophic forgetting
raised by the OOD problem in continual learning settings, and we named it
out-of-distribution forgetting (OODF). In continual image classification tasks,
we found that for a given category, introducing an intra-class distribution
shift significantly impaired the recognition accuracy of CL methods for that
category during subsequent learning. Interestingly, this phenomenon is special
for CL as the same level of distribution shift had only negligible effects in
the joint learning scenario. We verified that CL methods without dedicating
subnetworks for individual tasks are all vulnerable to OODF. Moreover, OODF
does not depend on any specific way of shifting the distribution, suggesting it
is a risk for CL in a wide range of circumstances. Taken together, our work
identified an under-attended risk during CL, highlighting the importance of
developing approaches that can overcome OODF.