Structural Knowledge Informed Continual Multivariate Time Series
  Forecasting
Zijie Pan Yushan Jiang Dongjin Song Sahil Garg Kashif Rasul Anderson Schneider Yuriy Nevmyvaka
Recent studies in multivariate time series (MTS) forecasting reveal that
explicitly modeling the hidden dependencies among different time series can
yield promising forecasting performance and reliable explanations. However,
modeling variable dependencies remains underexplored when MTS is continuously
accumulated under different regimes (stages). Due to the potential distribution
and dependency disparities, the underlying model may encounter the catastrophic
forgetting problem, i.e., it is challenging to memorize and infer different
types of variable dependencies across different regimes while maintaining
forecasting performance. To address this issue, we propose a novel Structural
Knowledge Informed Continual Learning (SKI-CL) framework to perform MTS
forecasting within a continual learning paradigm, which leverages structural
knowledge to steer the forecasting model toward identifying and adapting to
different regimes, and selects representative MTS samples from each regime for
memory replay. Specifically, we develop a forecasting model based on graph
structure learning, where a consistency regularization scheme is imposed
between the learned variable dependencies and the structural knowledge while
optimizing the forecasting objective over the MTS data. As such, MTS
representations learned in each regime are associated with distinct structural
knowledge, which helps the model memorize a variety of conceivable scenarios
and results in accurate forecasts in the continual learning context. Meanwhile,
we develop a representation-matching memory replay scheme that maximizes the
temporal coverage of MTS data to efficiently preserve the underlying temporal
dynamics and dependency structures of each regime. Thorough empirical studies
on synthetic and real-world benchmarks validate SKI-CL's efficacy and
advantages over the state-of-the-art for continual MTS forecasting tasks.