HiMTM: Hierarchical Multi-Scale Masked Time Series Modeling for
  Long-Term Forecasting
Shubao Zhao Ming Jin Zhaoxiang Hou Chengyi Yang Zengxiang Li Qingsong Wen Yi Wang
Time series forecasting is crucial and challenging in the real world. The
recent surge in interest regarding time series foundation models, which cater
to a diverse array of downstream tasks, is noteworthy. However, existing
methods often overlook the multi-scale nature of time series, an aspect crucial
for precise forecasting. To bridge this gap, we propose HiMTM, a hierarchical
multi-scale masked time series modeling method designed for long-term
forecasting. Specifically, it comprises four integral components: (1)
hierarchical multi-scale transformer (HMT) to capture temporal information at
different scales; (2) decoupled encoder-decoder (DED) forces the encoder to
focus on feature extraction, while the decoder to focus on pretext tasks; (3)
multi-scale masked reconstruction (MMR) provides multi-stage supervision
signals for pre-training; (4) cross-scale attention fine-tuning (CSA-FT) to
capture dependencies between different scales for forecasting. Collectively,
these components enhance multi-scale feature extraction capabilities in masked
time series modeling and contribute to improved prediction accuracy. We conduct
extensive experiments on 7 mainstream datasets to prove that HiMTM has obvious
advantages over contemporary self-supervised and end-to-end learning methods.
The effectiveness of HiMTM is further showcased by its application in the
industry of natural gas demand forecasting.