Revealing the Power of Spatial-Temporal Masked Autoencoders in
  Multivariate Time Series Forecasting
Jiarui Sun Yujie Fan Chin-Chia Michael Yeh Wei Zhang Girish Chowdhary
Multivariate time series (MTS) forecasting involves predicting future time
series data based on historical observations. Existing research primarily
emphasizes the development of complex spatial-temporal models that capture
spatial dependencies and temporal correlations among time series variables
explicitly. However, recent advances have been impeded by challenges relating
to data scarcity and model robustness. To address these issues, we propose
Spatial-Temporal Masked Autoencoders (STMAE), an MTS forecasting framework that
leverages masked autoencoders to enhance the performance of spatial-temporal
baseline models. STMAE consists of two learning stages. In the pretraining
stage, an encoder-decoder architecture is employed. The encoder processes the
partially visible MTS data produced by a novel dual-masking strategy, including
biased random walk-based spatial masking and patch-based temporal masking.
Subsequently, the decoders aim to reconstruct the masked counterparts from both
spatial and temporal perspectives. The pretraining stage establishes a
challenging pretext task, compelling the encoder to learn robust
spatial-temporal patterns. In the fine-tuning stage, the pretrained encoder is
retained, and the original decoder from existing spatial-temporal models is
appended for forecasting. Extensive experiments are conducted on multiple MTS
benchmarks. The promising results demonstrate that integrating STMAE into
various spatial-temporal models can largely enhance their MTS forecasting
capability.