Distillation Enhanced Time Series Forecasting Network with Momentum
  Contrastive Learning
Haozhi Gao Qianqian Ren Jinbao Li
Contrastive representation learning is crucial in time series analysis as it
alleviates the issue of data noise and incompleteness as well as sparsity of
supervision signal. However, existing constrastive learning frameworks usually
focus on intral-temporal features, which fails to fully exploit the intricate
nature of time series data. To address this issue, we propose DE-TSMCL, an
innovative distillation enhanced framework for long sequence time series
forecasting. Specifically, we design a learnable data augmentation mechanism
which adaptively learns whether to mask a timestamp to obtain optimized
sub-sequences. Then, we propose a contrastive learning task with momentum
update to explore inter-sample and intra-temporal correlations of time series
to learn the underlying structure feature on the unlabeled time series.
Meanwhile, we design a supervised task to learn more robust representations and
facilitate the contrastive learning process. Finally, we jointly optimize the
above two tasks. By developing model loss from multiple tasks, we can learn
effective representations for downstream forecasting task. Extensive
experiments, in comparison with state-of-the-arts, well demonstrate the
effectiveness of DE-TSMCL, where the maximum improvement can reach to 27.3%.