Cross-Corpus Multilingual Speech Emotion Recognition: Amharic vs. Other
  Languages
Ephrem Afele Retta Richard Sutcliffe Jabar Mahmood Michael Abebe Berwo Eiad Almekhlafi Sajjad Ahmed Khan Shehzad Ashraf Chaudhry Mustafa Mhamed Jun Feng
In a conventional Speech emotion recognition (SER) task, a classifier for a
given language is trained on a pre-existing dataset for that same language.
However, where training data for a language does not exist, data from other
languages can be used instead. We experiment with cross-lingual and
multilingual SER, working with Amharic, English, German and URDU. For Amharic,
we use our own publicly-available Amharic Speech Emotion Dataset (ASED). For
English, German and Urdu we use the existing RAVDESS, EMO-DB and URDU datasets.
We followed previous research in mapping labels for all datasets to just two
classes, positive and negative. Thus we can compare performance on different
languages directly, and combine languages for training and testing. In
Experiment 1, monolingual SER trials were carried out using three classifiers,
AlexNet, VGGE (a proposed variant of VGG), and ResNet50. Results averaged for
the three models were very similar for ASED and RAVDESS, suggesting that
Amharic and English SER are equally difficult. Similarly, German SER is more
difficult, and Urdu SER is easier. In Experiment 2, we trained on one language
and tested on another, in both directions for each pair: Amharic<->German,
Amharic<->English, and Amharic<->Urdu. Results with Amharic as target suggested
that using English or German as source will give the best result. In Experiment
3, we trained on several non-Amharic languages and then tested on Amharic. The
best accuracy obtained was several percent greater than the best accuracy in
Experiment 2, suggesting that a better result can be obtained when using two or
three non-Amharic languages for training than when using just one non-Amharic
language. Overall, the results suggest that cross-lingual and multilingual
training can be an effective strategy for training a SER classifier when
resources for a language are scarce.