Improving Acoustic Word Embeddings through Correspondence Training of
  Self-supervised Speech Representations
Amit Meghanani Thomas Hain
Acoustic word embeddings (AWEs) are vector representations of spoken words.
An effective method for obtaining AWEs is the Correspondence Auto-Encoder
(CAE). In the past, the CAE method has been associated with traditional MFCC
features. Representations obtained from self-supervised learning (SSL)-based
speech models such as HuBERT, Wav2vec2, etc., are outperforming MFCC in many
downstream tasks. However, they have not been well studied in the context of
learning AWEs. This work explores the effectiveness of CAE with SSL-based
speech representations to obtain improved AWEs. Additionally, the capabilities
of SSL-based speech models are explored in cross-lingual scenarios for
obtaining AWEs. Experiments are conducted on five languages: Polish,
Portuguese, Spanish, French, and English. HuBERT-based CAE model achieves the
best results for word discrimination in all languages, despite Hu-BERT being
pre-trained on English only. Also, the HuBERT-based CAE model works well in
cross-lingual settings. It outperforms MFCC-based CAE models trained on the
target languages when trained on one source language and tested on target
languages.