Curricular and Cyclical Loss for Time Series Learning Strategy
Chenxi Sun Hongyan Li Moxian Song Derun Cai Shenda Hong
Time series widely exists in real-world applications and many deep learning
models have performed well on it. Current research has shown the importance of
learning strategy for models, suggesting that the benefit is the order and size
of learning samples. However, no effective strategy has been proposed for time
series due to its abstract and dynamic construction. Meanwhile, the existing
one-shot tasks and continuous tasks for time series necessitate distinct
learning processes and mechanisms. No all-purpose approach has been suggested.
In this work, we propose a novel Curricular and CyclicaL loss (CRUCIAL) to
learn time series for the first time. It is model- and task-agnostic and can be
plugged on top of the original loss with no extra procedure. CRUCIAL has two
characteristics: It can arrange an easy-to-hard learning order by dynamically
determining the sample contribution and modulating the loss amplitude; It can
manage a cyclically changed dataset and achieve an adaptive cycle by
correlating the loss distribution and the selection probability. We prove that
compared with monotonous size, cyclical size can reduce expected error.
Experiments on 3 kinds of tasks and 5 real-world datasets show the benefits of
CRUCIAL for most deep learning models when learning time series.