Opening the black box of language acquisition
Jérôme Michaud Anna Jon-and
Recent advances in large language models using deep learning techniques have
renewed interest on how languages can be learned from data. However, it is
unclear whether or how these models represent grammatical information from the
learned languages. In addition, the models must be pre-trained on large corpora
before they can be used. In this work, we propose an alternative, more
transparent and cognitively plausible architecture for learning language.
Instead of using deep learning, our approach uses a minimal cognitive
architecture based on sequence memory and chunking. The learning mechanism is
based on the principles of reinforcement learning. We test our architecture on
a number of natural-like toy languages. Results show that the model can learn
these artificial languages from scratch and extract grammatical information
that supports learning. Our study demonstrates the power of this simple
architecture and stresses the importance of sequence memory as a key component
of the language learning process. Since other animals do not seem to have a
faithful sequence memory, this may explain why only humans have developed
complex languages.