Masked Autoencoders are Efficient Class Incremental Learners
Jiang-Tian Zhai Xialei Liu Andrew D. Bagdanov Ke Li Ming-Ming Cheng
Class Incremental Learning (CIL) aims to sequentially learn new classes while
avoiding catastrophic forgetting of previous knowledge. We propose to use
Masked Autoencoders (MAEs) as efficient learners for CIL. MAEs were originally
designed to learn useful representations through reconstructive unsupervised
learning, and they can be easily integrated with a supervised loss for
classification. Moreover, MAEs can reliably reconstruct original input images
from randomly selected patches, which we use to store exemplars from past tasks
more efficiently for CIL. We also propose a bilateral MAE framework to learn
from image-level and embedding-level fusion, which produces better-quality
reconstructed images and more stable representations. Our experiments confirm
that our approach performs better than the state-of-the-art on CIFAR-100,
ImageNet-Subset, and ImageNet-Full. The code is available at
https://github.com/scok30/MAE-CIL .