ARM: Refining Multivariate Forecasting with Adaptive Temporal-Contextual
  Learning
Jiecheng Lu Xu Han Shihao Yang
Long-term time series forecasting (LTSF) is important for various domains but
is confronted by challenges in handling the complex temporal-contextual
relationships. As multivariate input models underperforming some recent
univariate counterparts, we posit that the issue lies in the inefficiency of
existing multivariate LTSF Transformers to model series-wise relationships: the
characteristic differences between series are often captured incorrectly. To
address this, we introduce ARM: a multivariate temporal-contextual adaptive
learning method, which is an enhanced architecture specifically designed for
multivariate LTSF modelling. ARM employs Adaptive Univariate Effect Learning
(AUEL), Random Dropping (RD) training strategy, and Multi-kernel Local
Smoothing (MKLS), to better handle individual series temporal patterns and
correctly learn inter-series dependencies. ARM demonstrates superior
performance on multiple benchmarks without significantly increasing
computational costs compared to vanilla Transformer, thereby advancing the
state-of-the-art in LTSF. ARM is also generally applicable to other LTSF
architecture beyond vanilla Transformer.