Density Distribution-based Learning Framework for Addressing Online
  Continual Learning Challenges
Shilin Zhang Jiahui Wang
In this paper, we address the challenges of online Continual Learning (CL) by
introducing a density distribution-based learning framework. CL, especially the
Class Incremental Learning, enables adaptation to new test distributions while
continuously learning from a single-pass training data stream, which is more in
line with the practical application requirements of real-world scenarios.
However, existing CL methods often suffer from catastrophic forgetting and
higher computing costs due to complex algorithm designs, limiting their
practical use. Our proposed framework overcomes these limitations by achieving
superior average accuracy and time-space efficiency, bridging the performance
gap between CL and classical machine learning. Specifically, we adopt an
independent Generative Kernel Density Estimation (GKDE) model for each CL task.
During the testing stage, the GKDEs utilize a self-reported max probability
density value to determine which one is responsible for predicting incoming
test instances. A GKDE-based learning objective can ensure that samples with
the same label are grouped together, while dissimilar instances are pushed
farther apart. Extensive experiments conducted on multiple CL datasets validate
the effectiveness of our proposed framework. Our method outperforms popular CL
approaches by a significant margin, while maintaining competitive time-space
efficiency, making our framework suitable for real-world applications. Code
will be available at https://github.com/xxxx/xxxx.