Transformers, Contextualism, and Polysemy
Jumbly Grindrod
The transformer architecture, introduced by Vaswani et al. (2017), is at the
heart of the remarkable recent progress in the development of language models,
including famous chatbots such as Chat-gpt and Bard. In this paper, I argue
that we an extract from the way the transformer architecture works a picture of
the relationship between context and meaning. I call this the transformer
picture, and I argue that it is a novel with regard to two related
philosophical debates: the contextualism debate regarding the extent of
context-sensitivity across natural language, and the polysemy debate regarding
how polysemy should be captured within an account of word meaning. Although
much of the paper merely tries to position the transformer picture with respect
to these two debates, I will also begin to make the case for the transformer
picture.