Emergent Word Order Universals from Cognitively-Motivated Language
  Models
Tatsuki Kuribayashi Ryo Ueda Ryo Yoshida Yohei Oseki Ted Briscoe Timothy Baldwin
The world's languages exhibit certain so-called typological or implicational
universals; for example, Subject-Object-Verb (SOV) word order typically employs
postpositions. Explaining the source of such biases is a key goal in
linguistics. We study the word-order universals through a computational
simulation with language models (LMs). Our experiments show that typologically
typical word orders tend to have lower perplexity estimated by LMs with
cognitively plausible biases: syntactic biases, specific parsing strategies,
and memory limitations. This suggests that the interplay of these cognitive
biases and predictability (perplexity) can explain many aspects of word-order
universals. This also showcases the advantage of cognitively-motivated LMs,
which are typically employed in cognitive modeling, in the computational
simulation of language universals.