PILOT: A Pre-Trained Model-Based Continual Learning Toolbox
Hai-Long Sun Da-Wei Zhou Han-Jia Ye De-Chuan Zhan
While traditional machine learning can effectively tackle a wide range of
problems, it primarily operates within a closed-world setting, which presents
limitations when dealing with streaming data. As a solution, incremental
learning emerges to address real-world scenarios involving new data's arrival.
Recently, pre-training has made significant advancements and garnered the
attention of numerous researchers. The strong performance of these pre-trained
models (PTMs) presents a promising avenue for developing continual learning
algorithms that can effectively adapt to real-world scenarios. Consequently,
exploring the utilization of PTMs in incremental learning has become essential.
This paper introduces a pre-trained model-based continual learning toolbox
known as PILOT. On the one hand, PILOT implements some state-of-the-art
class-incremental learning algorithms based on pre-trained models, such as L2P,
DualPrompt, and CODA-Prompt. On the other hand, PILOT also fits typical
class-incremental learning algorithms (e.g., DER, FOSTER, and MEMO) within the
context of pre-trained models to evaluate their effectiveness.