Evaluating the structure of cognitive tasks with transfer learning
Bruno Aristimunha Raphael Y. de Camargo Walter H. Lopez Pinaya Sylvain Chevallier Alexandre Gramfort Cedric Rommel
Electroencephalography (EEG) decoding is a challenging task due to the
limited availability of labelled data. While transfer learning is a promising
technique to address this challenge, it assumes that transferable data domains
and task are known, which is not the case in this setting. This study
investigates the transferability of deep learning representations between
different EEG decoding tasks. We conduct extensive experiments using
state-of-the-art decoding models on two recently released EEG datasets, ERP
CORE and M$^3$CV, containing over 140 subjects and 11 distinct cognitive tasks.
We measure the transferability of learned representations by pre-training deep
neural networks on one task and assessing their ability to decode subsequent
tasks. Our experiments demonstrate that, even with linear probing transfer,
significant improvements in decoding performance can be obtained, with gains of
up to 28% compare with the pure supervised approach. Additionally, we discover
evidence that certain decoding paradigms elicit specific and narrow brain
activities, while others benefit from pre-training on a broad range of
representations. By revealing which tasks transfer well and demonstrating the
benefits of transfer learning for EEG decoding, our findings have practical
implications for mitigating data scarcity in this setting. The transfer maps
generated also provide insights into the hierarchical relations between
cognitive tasks, hence enhancing our understanding of how these tasks are
connected from a neuroscientific standpoint.