Class-incremental Learning for Time Series: Benchmark and Evaluation
Zhongzheng Qiao Quang Pham Zhen Cao Hoang H Le P. N. Suganthan Xudong Jiang Ramasamy Savitha
Real-world environments are inherently non-stationary, frequently introducing
new classes over time. This is especially common in time series classification,
such as the emergence of new disease classification in healthcare or the
addition of new activities in human activity recognition. In such cases, a
learning system is required to assimilate novel classes effectively while
avoiding catastrophic forgetting of the old ones, which gives rise to the
Class-incremental Learning (CIL) problem. However, despite the encouraging
progress in the image and language domains, CIL for time series data remains
relatively understudied. Existing studies suffer from inconsistent experimental
designs, necessitating a comprehensive evaluation and benchmarking of methods
across a wide range of datasets. To this end, we first present an overview of
the Time Series Class-incremental Learning (TSCIL) problem, highlight its
unique challenges, and cover the advanced methodologies. Further, based on
standardized settings, we develop a unified experimental framework that
supports the rapid development of new algorithms, easy integration of new
datasets, and standardization of the evaluation process. Using this framework,
we conduct a comprehensive evaluation of various generic and
time-series-specific CIL methods in both standard and privacy-sensitive
scenarios. Our extensive experiments not only provide a standard baseline to
support future research but also shed light on the impact of various design
factors such as normalization layers or memory budget thresholds. Codes are
available at https://github.com/zqiao11/TSCIL.