Mapping Brains with Language Models: A Survey
Antonia Karamolegkou Mostafa Abdou Anders Søgaard
Over the years, many researchers have seemingly made the same observation:
Brain and language model activations exhibit some structural similarities,
enabling linear partial mappings between features extracted from neural
recordings and computational language models. In an attempt to evaluate how
much evidence has been accumulated for this observation, we survey over 30
studies spanning 10 datasets and 8 metrics. How much evidence has been
accumulated, and what, if anything, is missing before we can draw conclusions?
Our analysis of the evaluation methods used in the literature reveals that some
of the metrics are less conservative. We also find that the accumulated
evidence, for now, remains ambiguous, but correlations with model size and
quality provide grounds for cautious optimism.