Large Pre-trained time series models for cross-domain Time series
  analysis tasks
Harshavardhan Kamarthi B. Aditya Prakash
Large pre-trained models have been instrumental in significant advancements
in domains like language and vision making model training for individual
downstream tasks more efficient as well as provide superior performance.
However, tackling time-series analysis tasks usually involves designing and
training a separate model from scratch leveraging training data and domain
expertise specific to the task. We tackle a significant challenge for
pre-training a general time-series model from multiple heterogeneous
time-series dataset: providing semantically useful inputs to models for
modeling time series of different dynamics from different domains. We observe
that partitioning time-series into segments as inputs to sequential models
produces semantically better inputs and propose a novel model LPTM that
automatically identifies optimal dataset-specific segmentation strategy
leveraging self-supervised learning loss during pre-training. LPTM provides
performance similar to or better than domain-specific state-of-art model and is
significantly more data and compute efficient taking up to 40% less data as
well as 50% less training time to achieve state-of-art performance in a wide
range of time-series analysis tasks from multiple disparate domain.