RWKV-TS: Beyond Traditional Recurrent Neural Network for Time Series
  Tasks
Haowen Hou F. Richard Yu
Traditional Recurrent Neural Network (RNN) architectures, such as LSTM and
GRU, have historically held prominence in time series tasks. However, they have
recently seen a decline in their dominant position across various time series
tasks. As a result, recent advancements in time series forecasting have seen a
notable shift away from RNNs towards alternative architectures such as
Transformers, MLPs, and CNNs. To go beyond the limitations of traditional RNNs,
we design an efficient RNN-based model for time series tasks, named RWKV-TS,
with three distinctive features: (i) A novel RNN architecture characterized by
$O(L)$ time complexity and memory usage. (ii) An enhanced ability to capture
long-term sequence information compared to traditional RNNs. (iii) High
computational efficiency coupled with the capacity to scale up effectively.
Through extensive experimentation, our proposed RWKV-TS model demonstrates
competitive performance when compared to state-of-the-art Transformer-based or
CNN-based models. Notably, RWKV-TS exhibits not only comparable performance but
also demonstrates reduced latency and memory utilization. The success of
RWKV-TS encourages further exploration and innovation in leveraging RNN-based
approaches within the domain of Time Series. The combination of competitive
performance, low latency, and efficient memory usage positions RWKV-TS as a
promising avenue for future research in time series tasks. Code is available
at:\href{https://github.com/howard-hou/RWKV-TS}{
https://github.com/howard-hou/RWKV-TS}