Spatial HuBERT: Self-supervised Spatial Speech Representation Learning
  for a Single Talker from Multi-channel Audio
Antoni Dimitriadis Siqi Pan Vidhyasaharan Sethu Beena Ahmed
Self-supervised learning has been used to leverage unlabelled data, improving
accuracy and generalisation of speech systems through the training of
representation models. While many recent works have sought to produce effective
representations across a variety of acoustic domains, languages, modalities and
even simultaneous speakers, these studies have all been limited to
single-channel audio recordings. This paper presents Spatial HuBERT, a
self-supervised speech representation model that learns both acoustic and
spatial information pertaining to a single speaker in a potentially noisy
environment by using multi-channel audio inputs. Spatial HuBERT learns
representations that outperform state-of-the-art single-channel speech
representations on a variety of spatial downstream tasks, particularly in
reverberant and noisy environments. We also demonstrate the utility of the
representations learned by Spatial HuBERT on a speech localisation downstream
task. Along with this paper, we publicly release a new dataset of 100 000
simulated first-order ambisonics room impulse responses.