LSTTN: A Long-Short Term Transformer-based Spatio-temporal Neural
  Network for Traffic Flow Forecasting
Qinyao Luo Silu He Xing Han Yuhan Wang Haifeng Li
Accurate traffic forecasting is a fundamental problem in intelligent
transportation systems and learning long-range traffic representations with key
information through spatiotemporal graph neural networks (STGNNs) is a basic
assumption of current traffic flow prediction models. However, due to
structural limitations, existing STGNNs can only utilize short-range traffic
flow data; therefore, the models cannot adequately learn the complex trends and
periodic features in traffic flow. Besides, it is challenging to extract the
key temporal information from the long historical traffic series and obtain a
compact representation. To solve the above problems, we propose a novel LSTTN
(Long-Short Term Transformer-based Network) framework comprehensively
considering the long- and short-term features in historical traffic flow.
First, we employ a masked subseries Transformer to infer the content of masked
subseries from a small portion of unmasked subseries and their temporal context
in a pretraining manner, forcing the model to efficiently learn compressed and
contextual subseries temporal representations from long historical series.
Then, based on the learned representations, long-term trend is extracted by
using stacked 1D dilated convolution layers, and periodic features are
extracted by dynamic graph convolution layers. For the difficulties in making
time-step level prediction, LSTTN adopts a short-term trend extractor to learn
fine-grained short-term temporal features. Finally, LSTTN fuses the long-term
trend, periodic features and short-term features to obtain the prediction
results. Experiments on four real-world datasets show that in 60-minute-ahead
long-term forecasting, the LSTTN model achieves a minimum improvement of 5.63\%
and a maximum improvement of 16.78\% over baseline models. The source code is
available at https://github.com/GeoX-Lab/LSTTN.