GBT: Two-stage transformer framework for non-stationary time series
  forecasting
Li Shen Yuning Wei Yangzhu Wang
This paper shows that time series forecasting Transformer (TSFT) suffers from
severe over-fitting problem caused by improper initialization method of unknown
decoder inputs, esp. when handling non-stationary time series. Based on this
observation, we propose GBT, a novel two-stage Transformer framework with Good
Beginning. It decouples the prediction process of TSFT into two stages,
including Auto-Regression stage and Self-Regression stage to tackle the problem
of different statistical properties between input and prediction
sequences.Prediction results of Auto-Regression stage serve as a Good
Beginning, i.e., a better initialization for inputs of Self-Regression stage.
We also propose Error Score Modification module to further enhance the
forecasting capability of the Self-Regression stage in GBT. Extensive
experiments on seven benchmark datasets demonstrate that GBT outperforms SOTA
TSFTs (FEDformer, Pyraformer, ETSformer, etc.) and many other forecasting
models (SCINet, N-HiTS, etc.) with only canonical attention and convolution
while owning less time and space complexity. It is also general enough to
couple with these models to strengthen their forecasting capability. The source
code is available at: https://github.com/OrigamiSL/GBT