Take an Irregular Route: Enhance the Decoder of Time-Series Forecasting
  Transformer
Li Shen Yuning Wei Yangzhu Wang Hongguang Li
With the development of Internet of Things (IoT) systems, precise long-term
forecasting method is requisite for decision makers to evaluate current
statuses and formulate future policies. Currently, Transformer and MLP are two
paradigms for deep time-series forecasting and the former one is more
prevailing in virtue of its exquisite attention mechanism and encoder-decoder
architecture. However, data scientists seem to be more willing to dive into the
research of encoder, leaving decoder unconcerned. Some researchers even adopt
linear projections in lieu of the decoder to reduce the complexity. We argue
that both extracting the features of input sequence and seeking the relations
of input and prediction sequence, which are respective functions of encoder and
decoder, are of paramount significance. Motivated from the success of FPN in CV
field, we propose FPPformer to utilize bottom-up and top-down architectures
respectively in encoder and decoder to build the full and rational hierarchy.
The cutting-edge patch-wise attention is exploited and further developed with
the combination, whose format is also different in encoder and decoder, of
revamped element-wise attention in this work. Extensive experiments with six
state-of-the-art baselines on twelve benchmarks verify the promising
performances of FPPformer and the importance of elaborately devising decoder in
time-series forecasting Transformer. The source code is released in
https://github.com/OrigamiSL/FPPformer.