Learning Robust and Consistent Time Series Representations: A Dilated
  Inception-Based Approach
Anh Duy Nguyen Trang H. Tran Hieu H. Pham Phi Le Nguyen Lam M. Nguyen
Representation learning for time series has been an important research area
for decades. Since the emergence of the foundation models, this topic has
attracted a lot of attention in contrastive self-supervised learning, to solve
a wide range of downstream tasks. However, there have been several challenges
for contrastive time series processing. First, there is no work considering
noise, which is one of the critical factors affecting the efficacy of time
series tasks. Second, there is a lack of efficient yet lightweight encoder
architectures that can learn informative representations robust to various
downstream tasks. To fill in these gaps, we initiate a novel sampling strategy
that promotes consistent representation learning with the presence of noise in
natural time series. In addition, we propose an encoder architecture that
utilizes dilated convolution within the Inception block to create a scalable
and robust network architecture with a wide receptive field. Experiments
demonstrate that our method consistently outperforms state-of-the-art methods
in forecasting, classification, and abnormality detection tasks, e.g. ranks
first over two-thirds of the classification UCR datasets, with only $40\%$ of
the parameters compared to the second-best approach. Our source code for
CoInception framework is accessible at
https://github.com/anhduy0911/CoInception.