What has LeBenchmark Learnt about French Syntax?
Zdravko Dugonjić Adrien Pupier Benjamin Lecouteux Maximin Coavoux
The paper reports on a series of experiments aiming at probing LeBenchmark, a
pretrained acoustic model trained on 7k hours of spoken French, for syntactic
information. Pretrained acoustic models are increasingly used for downstream
speech tasks such as automatic speech recognition, speech translation, spoken
language understanding or speech parsing. They are trained on very low level
information (the raw speech signal), and do not have explicit lexical
knowledge. Despite that, they obtained reasonable results on tasks that
requires higher level linguistic knowledge. As a result, an emerging question
is whether these models encode syntactic information. We probe each
representation layer of LeBenchmark for syntax, using the Orf\'eo treebank, and
observe that it has learnt some syntactic information. Our results show that
syntactic information is more easily extractable from the middle layers of the
network, after which a very sharp decrease is observed.