Singer Identity Representation Learning using Self-Supervised Techniques
Bernardo Torres Stefan Lattner Gaël Richard
Significant strides have been made in creating voice identity representations
using speech data. However, the same level of progress has not been achieved
for singing voices. To bridge this gap, we suggest a framework for training
singer identity encoders to extract representations suitable for various
singing-related tasks, such as singing voice similarity and synthesis. We
explore different self-supervised learning techniques on a large collection of
isolated vocal tracks and apply data augmentations during training to ensure
that the representations are invariant to pitch and content variations. We
evaluate the quality of the resulting representations on singer similarity and
identification tasks across multiple datasets, with a particular emphasis on
out-of-domain generalization. Our proposed framework produces high-quality
embeddings that outperform both speaker verification and wav2vec 2.0
pre-trained baselines on singing voice while operating at 44.1 kHz. We release
our code and trained models to facilitate further research on singing voice and
related areas.