Can We Understand Plasticity Through Neural Collapse?
Guglielmo Bonifazi Iason Chalas Gian Hess Jakub Łucki
This paper explores the connection between two recently identified phenomena
in deep learning: plasticity loss and neural collapse. We analyze their
correlation in different scenarios, revealing a significant association during
the initial training phase on the first task. Additionally, we introduce a
regularization approach to mitigate neural collapse, demonstrating its
effectiveness in alleviating plasticity loss in this specific setting.