Contrastive Learning of Shared Spatiotemporal EEG Representations Across
  Individuals for Naturalistic Neuroscience
Xinke Shen Lingyi Tao Xuyang Chen Sen Song Quanying Liu Dan Zhang
Neural representations induced by naturalistic stimuli offer insights into
how humans respond to peripheral stimuli in daily life. The key to
understanding the general neural mechanisms underlying naturalistic stimuli
processing involves aligning neural activities across individuals and
extracting inter-subject shared neural representations. Targeting the
Electroencephalogram (EEG) technique, known for its rich spatial and temporal
information, this study presents a general framework for Contrastive Learning
of Shared SpatioTemporal EEG Representations across individuals (CL-SSTER).
Harnessing the representational capabilities of contrastive learning, CL-SSTER
utilizes a neural network to maximize the similarity of EEG representations
across individuals for identical stimuli, contrasting with those for varied
stimuli. The network employed spatial and temporal convolutions to
simultaneously learn the spatial and temporal patterns inherent in EEG. The
versatility of CL-SSTER was demonstrated on three EEG datasets, including a
synthetic dataset, a speech audio EEG dataset, and an emotional video EEG
dataset. CL-SSTER attained the highest inter-subject correlation (ISC) values
compared to the state-of-the-art ISC methods. The latent representations
generated by CL-SSTER exhibited reliable spatiotemporal EEG patterns, which can
be explained by specific aspects of the stimuli. CL-SSTER serves as an
interpretable and scalable foundational framework for the identification of
inter-subject shared neural representations in the realm of naturalistic
neuroscience.