SLABERT Talk Pretty One Day: Modeling Second Language Acquisition with
  BERT
Aditya Yadavalli Alekhya Yadavalli Vera Tobin
Second language acquisition (SLA) research has extensively studied
cross-linguistic transfer, the influence of linguistic structure of a speaker's
native language [L1] on the successful acquisition of a foreign language [L2].
Effects of such transfer can be positive (facilitating acquisition) or negative
(impeding acquisition). We find that NLP literature has not given enough
attention to the phenomenon of negative transfer. To understand patterns of
both positive and negative transfer between L1 and L2, we model sequential
second language acquisition in LMs. Further, we build a Mutlilingual Age
Ordered CHILDES (MAO-CHILDES) -- a dataset consisting of 5 typologically
diverse languages, i.e., German, French, Polish, Indonesian, and Japanese -- to
understand the degree to which native Child-Directed Speech (CDS) [L1] can help
or conflict with English language acquisition [L2]. To examine the impact of
native CDS, we use the TILT-based cross lingual transfer learning approach
established by Papadimitriou and Jurafsky (2020) and find that, as in human
SLA, language family distance predicts more negative transfer. Additionally, we
find that conversational speech data shows greater facilitation for language
acquisition than scripted speech data. Our findings call for further research
using our novel Transformer-based SLA models and we would like to encourage it
by releasing our code, data, and models.