Language Models Can Learn Exceptions to Syntactic Rules
Cara Su-Yi Leong Tal Linzen
Artificial neural networks can generalize productively to novel contexts. Can
they also learn exceptions to those productive rules? We explore this question
using the case of restrictions on English passivization (e.g., the fact that
"The vacation lasted five days" is grammatical, but "*Five days was lasted by
the vacation" is not). We collect human acceptability judgments for passive
sentences with a range of verbs, and show that the probability distribution
defined by GPT-2, a language model, matches the human judgments with high
correlation. We also show that the relative acceptability of a verb in the
active vs. passive voice is positively correlated with the relative frequency
of its occurrence in those voices. These results provide preliminary support
for the entrenchment hypothesis, according to which learners track and uses the
distributional properties of their input to learn negative exceptions to rules.
At the same time, this hypothesis fails to explain the magnitude of
unpassivizability demonstrated by certain individual verbs, suggesting that
other cues to exceptionality are available in the linguistic input.