Score-balanced Loss for Multi-aspect Pronunciation Assessment
Heejin Do Yunsu Kim Gary Geunbae Lee
With rapid technological growth, automatic pronunciation assessment has
transitioned toward systems that evaluate pronunciation in various aspects,
such as fluency and stress. However, despite the highly imbalanced score labels
within each aspect, existing studies have rarely tackled the data imbalance
problem. In this paper, we suggest a novel loss function, score-balanced loss,
to address the problem caused by uneven data, such as bias toward the majority
scores. As a re-weighting approach, we assign higher costs when the predicted
score is of the minority class, thus, guiding the model to gain positive
feedback for sparse score prediction. Specifically, we design two weighting
factors by leveraging the concept of an effective number of samples and using
the ranks of scores. We evaluate our method on the speechocean762 dataset,
which has noticeably imbalanced scores for several aspects. Improved results
particularly on such uneven aspects prove the effectiveness of our method.