RITA: Group Attention is All You Need for Timeseries Analytics
Jiaming Liang Lei Cao Samuel Madden Zachary Ives Guoliang Li
Timeseries analytics is of great importance in many real-world applications.
Recently, the Transformer model, popular in natural language processing, has
been leveraged to learn high quality feature embeddings from timeseries, core
to the performance of various timeseries analytics tasks. However, the
quadratic time and space complexities limit Transformers' scalability,
especially for long timeseries. To address these issues, we develop a
timeseries analytics tool, RITA, which uses a novel attention mechanism, named
group attention, to address this scalability issue. Group attention dynamically
clusters the objects based on their similarity into a small number of groups
and approximately computes the attention at the coarse group granularity. It
thus significantly reduces the time and space complexity, yet provides a
theoretical guarantee on the quality of the computed attention. The dynamic
scheduler of RITA continuously adapts the number of groups and the batch size
in the training process, ensuring group attention always uses the fewest groups
needed to meet the approximation quality requirement. Extensive experiments on
various timeseries datasets and analytics tasks demonstrate that RITA
outperforms the state-of-the-art in accuracy and is significantly faster --
with speedups of up to 63X.