Establishing degrees of closeness between audio recordings along
  different dimensions using large-scale cross-lingual models
Maxime Fily Guillaume Wisniewski Severine Guillaume Gilles Adda Alexis Michaud
In the highly constrained context of low-resource language studies, we
explore vector representations of speech from a pretrained model to determine
their level of abstraction with regard to the audio signal. We propose a new
unsupervised method using ABX tests on audio recordings with carefully curated
metadata to shed light on the type of information present in the
representations. ABX tests determine whether the representations computed by a
multilingual speech model encode a given characteristic. Three experiments are
devised: one on room acoustics aspects, one on linguistic genre, and one on
phonetic aspects. The results confirm that the representations extracted from
recordings with different linguistic/extra-linguistic characteristics differ
along the same lines. Embedding more audio signal in one vector better
discriminates extra-linguistic characteristics, whereas shorter snippets are
better to distinguish segmental information. The method is fully unsupervised,
potentially opening new research avenues for comparative work on
under-documented languages.