Continual Learning and Catastrophic Forgetting
Gido M. van de Ven Nicholas Soures Dhireesha Kudithipudi
This book chapter delves into the dynamics of continual learning, which is
the process of incrementally learning from a non-stationary stream of data.
Although continual learning is a natural skill for the human brain, it is very
challenging for artificial neural networks. An important reason is that, when
learning something new, these networks tend to quickly and drastically forget
what they had learned before, a phenomenon known as catastrophic forgetting.
Especially in the last decade, continual learning has become an extensively
studied topic in deep learning. This book chapter reviews the insights that
this field has generated.