Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Everton L. Aleixo Juan G. Colonna Marco Cristo Everlandio Fernandes
Deep Learning models have achieved remarkable performance in tasks such as
image classification or generation, often surpassing human accuracy. However,
they can struggle to learn new tasks and update their knowledge without access
to previous data, leading to a significant loss of accuracy known as
Catastrophic Forgetting (CF). This phenomenon was first observed by McCloskey
and Cohen in 1989 and remains an active research topic. Incremental learning
without forgetting is widely recognized as a crucial aspect in building better
AI systems, as it allows models to adapt to new tasks without losing the
ability to perform previously learned ones. This article surveys recent studies
that tackle CF in modern Deep Learning models that use gradient descent as
their learning algorithm. Although several solutions have been proposed, a
definitive solution or consensus on assessing CF is yet to be established. The
article provides a comprehensive review of recent solutions, proposes a
taxonomy to organize them, and identifies research gaps in this area.