Improving End-to-End Speech Translation by Imitation-Based Knowledge
  Distillation with Synthetic Transcripts
Rebekka Hubert Artem Sokolov Stefan Riezler
End-to-end automatic speech translation (AST) relies on data that combines
audio inputs with text translation outputs. Previous work used existing large
parallel corpora of transcriptions and translations in a knowledge distillation
(KD) setup to distill a neural machine translation (NMT) into an AST student
model. While KD allows using larger pretrained models, the reliance of previous
KD approaches on manual audio transcripts in the data pipeline restricts the
applicability of this framework to AST. We present an imitation learning
approach where a teacher NMT system corrects the errors of an AST student
without relying on manual transcripts. We show that the NMT teacher can recover
from errors in automatic transcriptions and is able to correct erroneous
translations of the AST student, leading to improvements of about 4 BLEU points
over the standard AST end-to-end baseline on the English-German CoVoST-2 and
MuST-C datasets, respectively. Code and data are publicly
available.\footnote{\url{https://github.com/HubReb/imitkd_ast/releases/tag/v1.1}}