Considering Nonstationary within Multivariate Time Series with
  Variational Hierarchical Transformer for Forecasting
Muyao Wang Wenchao Chen Bo Chen
The forecasting of Multivariate Time Series (MTS) has long been an important
but challenging task. Due to the non-stationary problem across long-distance
time steps, previous studies primarily adopt stationarization method to
attenuate the non-stationary problem of the original series for better
predictability. However, existing methods always adopt the stationarized
series, which ignores the inherent non-stationarity, and has difficulty in
modeling MTS with complex distributions due to the lack of stochasticity. To
tackle these problems, we first develop a powerful hierarchical probabilistic
generative module to consider the non-stationarity and stochastic
characteristics within MTS, and then combine it with transformer for a
well-defined variational generative dynamic model named Hierarchical Time
series Variational Transformer (HTV-Trans), which recovers the intrinsic
non-stationary information into temporal dependencies. Being a powerful
probabilistic model, HTV-Trans is utilized to learn expressive representations
of MTS and applied to forecasting tasks. Extensive experiments on diverse
datasets show the efficiency of HTV-Trans on MTS forecasting tasks