EEG2Rep: Enhancing Self-supervised EEG Representation Through
  Informative Masked Inputs
Navid Mohammadi Foumani Geoffrey Mackellar Soheila Ghane Saad Irtza Nam Nguyen Mahsa Salehi
Self-supervised approaches for electroencephalography (EEG) representation
learning face three specific challenges inherent to EEG data: (1) The low
signal-to-noise ratio which challenges the quality of the representation
learned, (2) The wide range of amplitudes from very small to relatively large
due to factors such as the inter-subject variability, risks the models to be
dominated by higher amplitude ranges, and (3) The absence of explicit
segmentation in the continuous-valued sequences which can result in less
informative representations. To address these challenges, we introduce EEG2Rep,
a self-prediction approach for self-supervised representation learning from
EEG. Two core novel components of EEG2Rep are as follows: 1) Instead of
learning to predict the masked input from raw EEG, EEG2Rep learns to predict
masked input in latent representation space, and 2) Instead of conventional
masking methods, EEG2Rep uses a new semantic subsequence preserving (SSP)
method which provides informative masked inputs to guide EEG2Rep to generate
rich semantic representations. In experiments on 6 diverse EEG tasks with
subject variability, EEG2Rep significantly outperforms state-of-the-art
methods. We show that our semantic subsequence preserving improves the existing
masking methods in self-prediction literature and find that preserving 50\% of
EEG recordings will result in the most accurate results on all 6 tasks on
average. Finally, we show that EEG2Rep is robust to noise addressing a
significant challenge that exists in EEG data. Models and code are available
at: https://github.com/Navidfoumani/EEG2Rep