Acoustic characterization of speech rhythm: going beyond metrics with
  recurrent neural networks
François Deloche Laurent Bonnasse-Gahot Judit Gervain
Languages have long been described according to their perceived rhythmic
attributes. The associated typologies are of interest in psycholinguistics as
they partly predict newborns' abilities to discriminate between languages and
provide insights into how adult listeners process non-native languages. Despite
the relative success of rhythm metrics in supporting the existence of
linguistic rhythmic classes, quantitative studies have yet to capture the full
complexity of temporal regularities associated with speech rhythm. We argue
that deep learning offers a powerful pattern-recognition approach to advance
the characterization of the acoustic bases of speech rhythm. To explore this
hypothesis, we trained a medium-sized recurrent neural network on a language
identification task over a large database of speech recordings in 21 languages.
The network had access to the amplitude envelopes and a variable identifying
the voiced segments, assuming that this signal would poorly convey phonetic
information but preserve prosodic features. The network was able to identify
the language of 10-second recordings in 40% of the cases, and the language was
in the top-3 guesses in two-thirds of the cases. Visualization methods show
that representations built from the network activations are consistent with
speech rhythm typologies, although the resulting maps are more complex than two
separated clusters between stress and syllable-timed languages. We further
analyzed the model by identifying correlations between network activations and
known speech rhythm metrics. The findings illustrate the potential of deep
learning tools to advance our understanding of speech rhythm through the
identification and exploration of linguistically relevant acoustic feature
spaces.