ARNN: Attentive Recurrent Neural Network for Multi-channel EEG Signals
  to Identify Epileptic Seizures
Salim Rukhsar Anil Kumar Tiwari
We proposed an Attentive Recurrent Neural Network (ARNN), which recurrently
applies attention layers along a sequence and has linear complexity with
respect to the sequence length. The proposed model operates on multi-channel
EEG signals rather than single channel signals and leverages parallel
computation. In this cell, the attention layer is a computational unit that
efficiently applies self-attention and cross-attention mechanisms to compute a
recurrent function over a wide number of state vectors and input signals. Our
architecture is inspired in part by the attention layer and long short-term
memory (LSTM) cells, and it uses long-short style gates, but it scales this
typical cell up by several orders to parallelize for multi-channel EEG signals.
It inherits the advantages of attention layers and LSTM gate while avoiding
their respective drawbacks. We evaluated the model effectiveness through
extensive experiments with heterogeneous datasets, including the CHB-MIT and
UPenn and Mayos Clinic, CHB-MIT datasets. The empirical findings suggest that
the ARNN model outperforms baseline methods such as LSTM, Vision Transformer
(ViT), Compact Convolution Transformer (CCT), and R-Transformer (RT),
showcasing superior performance and faster processing capabilities across a
wide range of tasks. The code has been made publicly accessible at
\url{https://github.com/Salim-Lysiun/ARNN}.