Anaphoric Structure Emerges Between Neural Networks
Nicholas Edwards Hannah Rohde Henry Conklin
Pragmatics is core to natural language, enabling speakers to communicate
efficiently with structures like ellipsis and anaphora that can shorten
utterances without loss of meaning. These structures require a listener to
interpret an ambiguous form - like a pronoun - and infer the speaker's intended
meaning - who that pronoun refers to. Despite potential to introduce ambiguity,
anaphora is ubiquitous across human language. In an effort to better understand
the origins of anaphoric structure in natural language, we look to see if
analogous structures can emerge between artificial neural networks trained to
solve a communicative task. We show that: first, despite the potential for
increased ambiguity, languages with anaphoric structures are learnable by
neural models. Second, anaphoric structures emerge between models 'naturally'
without need for additional constraints. Finally, introducing an explicit
efficiency pressure on the speaker increases the prevalence of these
structures. We conclude that certain pragmatic structures straightforwardly
emerge between neural networks, without explicit efficiency pressures, but that
the competing needs of speakers and listeners conditions the degree and nature
of their emergence.