Wake-Sleep Consolidated Learning
Amelia Sorrenti Giovanni Bellitto Federica Proietto Salanitri Matteo Pennisi Simone Palazzo Concetto Spampinato
We propose Wake-Sleep Consolidated Learning (WSCL), a learning strategy
leveraging Complementary Learning System theory and the wake-sleep phases of
the human brain to improve the performance of deep neural networks for visual
classification tasks in continual learning settings. Our method learns
continually via the synchronization between distinct wake and sleep phases.
During the wake phase, the model is exposed to sensory input and adapts its
representations, ensuring stability through a dynamic parameter freezing
mechanism and storing episodic memories in a short-term temporary memory
(similarly to what happens in the hippocampus). During the sleep phase, the
training process is split into NREM and REM stages. In the NREM stage, the
model's synaptic weights are consolidated using replayed samples from the
short-term and long-term memory and the synaptic plasticity mechanism is
activated, strengthening important connections and weakening unimportant ones.
In the REM stage, the model is exposed to previously-unseen realistic visual
sensory experience, and the dreaming process is activated, which enables the
model to explore the potential feature space, thus preparing synapses to future
knowledge. We evaluate the effectiveness of our approach on three benchmark
datasets: CIFAR-10, Tiny-ImageNet and FG-ImageNet. In all cases, our method
outperforms the baselines and prior work, yielding a significant performance
gain on continual visual classification tasks. Furthermore, we demonstrate the
usefulness of all processing stages and the importance of dreaming to enable
positive forward transfer.