Improving Metrics for Speech Translation
Claudio Paonessa Dominik Frefel Manfred Vogel
We introduce Parallel Paraphrasing ($\text{Para}_\text{both}$), an
augmentation method for translation metrics making use of automatic
paraphrasing of both the reference and hypothesis. This method counteracts the
typically misleading results of speech translation metrics such as WER, CER,
and BLEU if only a single reference is available. We introduce two new datasets
explicitly created to measure the quality of metrics intended to be applied to
Swiss German speech-to-text systems. Based on these datasets, we show that we
are able to significantly improve the correlation with human quality perception
if our method is applied to commonly used metrics.