Multi-scale Transformer Pyramid Networks for Multivariate Time Series
  Forecasting
Yifan Zhang Rui Wu Sergiu M. Dascalu Frederick C. Harris Jr
Multivariate Time Series (MTS) forecasting involves modeling temporal
dependencies within historical records. Transformers have demonstrated
remarkable performance in MTS forecasting due to their capability to capture
long-term dependencies. However, prior work has been confined to modeling
temporal dependencies at either a fixed scale or multiple scales that
exponentially increase (most with base 2). This limitation hinders their
effectiveness in capturing diverse seasonalities, such as hourly and daily
patterns. In this paper, we introduce a dimension invariant embedding technique
that captures short-term temporal dependencies and projects MTS data into a
higher-dimensional space, while preserving the dimensions of time steps and
variables in MTS data. Furthermore, we present a novel Multi-scale Transformer
Pyramid Network (MTPNet), specifically designed to effectively capture temporal
dependencies at multiple unconstrained scales. The predictions are inferred
from multi-scale latent representations obtained from transformers at various
scales. Extensive experiments on nine benchmark datasets demonstrate that the
proposed MTPNet outperforms recent state-of-the-art methods.