S-JEPA: towards seamless cross-dataset transfer through dynamic spatial
  attention
Pierre Guetschel Thomas Moreau Michael Tangermann
Motivated by the challenge of seamless cross-dataset transfer in EEG signal
processing, this article presents an exploratory study on the use of Joint
Embedding Predictive Architectures (JEPAs). In recent years, self-supervised
learning has emerged as a promising approach for transfer learning in various
domains. However, its application to EEG signals remains largely unexplored. In
this article, we introduce Signal-JEPA for representing EEG recordings which
includes a novel domain-specific spatial block masking strategy and three novel
architectures for downstream classification. The study is conducted on a
54~subjects dataset and the downstream performance of the models is evaluated
on three different BCI paradigms: motor imagery, ERP and SSVEP. Our study
provides preliminary evidence for the potential of JEPAs in EEG signal
encoding. Notably, our results highlight the importance of spatial filtering
for accurate downstream classification and reveal an influence of the length of
the pre-training examples but not of the mask size on the downstream
performance.