Tuning In to Neural Encoding: Linking Human Brain and Artificial
  Supervised Representations of Language
Jingyuan Sun Xiaohan Zhang Marie-Francine Moens
To understand the algorithm that supports the human brain's language
representation, previous research has attempted to predict neural responses to
linguistic stimuli using embeddings generated by artificial neural networks
(ANNs), a process known as neural encoding. However, most of these studies have
focused on probing neural representations of Germanic languages, such as
English, with unsupervised ANNs. In this paper, we propose to bridge the gap
between human brain and supervised ANN representations of the Chinese language.
Specifically, we investigate how task tuning influences a pretained Transformer
for neural encoding and which tasks lead to the best encoding performances. We
generate supervised representations on eight Natural Language Understanding
(NLU) tasks using prompt-tuning, a technique that is seldom explored in neural
encoding for language. We demonstrate that prompt-tuning yields representations
that better predict neural responses to Chinese stimuli than traditional
fine-tuning on four tasks. Furthermore, we discover that tasks that require a
fine-grained processing of concepts and entities lead to representations that
are most predictive of brain activation patterns. Additionally, we reveal that
the proportion of tuned parameters highly influences the neural encoding
performance of fine-tuned models. Overall, our experimental findings could help
us better understand the relationship between supervised artificial and brain
language representations.