Analysis of Levenshtein Transformer's Decoder and Its Variants
Ruiyang Zhou
Levenshtein transformer (LevT) is a non-autoregressive machine translation
model with high decoding efficiency and comparable translation quality in terms
of bleu score, due to its parallel decoding and iterative refinement procedure.
Are there any deficiencies of its translations and what improvements could be
made? In this report, we focus on LevT's decoder and analyse the decoding
results length, subword generation, and deletion module's capability. We hope
to identify weaknesses of the decoder for future improvements.
  We also compare translations of the original LevT, knowledge-distilled LevT,
LevT with translation memory, and the KD-LevT with translation memory to see
how KD and translation memory can help.