Wave to Syntax: Probing spoken language models for syntax
Gaofei Shen Afra Alishahi Arianna Bisazza Grzegorz Chrupała
Understanding which information is encoded in deep models of spoken and
written language has been the focus of much research in recent years, as it is
crucial for debugging and improving these architectures. Most previous work has
focused on probing for speaker characteristics, acoustic and phonological
information in models of spoken language, and for syntactic information in
models of written language. Here we focus on the encoding of syntax in several
self-supervised and visually grounded models of spoken language. We employ two
complementary probing methods, combined with baselines and reference
representations to quantify the degree to which syntactic structure is encoded
in the activations of the target models. We show that syntax is captured most
prominently in the middle layers of the networks, and more explicitly within
models with more parameters.