Enhancing Supervised Learning with Contrastive Markings in Neural
  Machine Translation Training
Nathaniel Berger Miriam Exel Matthias Huck Stefan Riezler
Supervised learning in Neural Machine Translation (NMT) typically follows a
teacher forcing paradigm where reference tokens constitute the conditioning
context in the model's prediction, instead of its own previous predictions. In
order to alleviate this lack of exploration in the space of translations, we
present a simple extension of standard maximum likelihood estimation by a
contrastive marking objective. The additional training signals are extracted
automatically from reference translations by comparing the system hypothesis
against the reference, and used for up/down-weighting correct/incorrect tokens.
The proposed new training procedure requires one additional translation pass
over the training set per epoch, and does not alter the standard inference
setup. We show that training with contrastive markings yields improvements on
top of supervised learning, and is especially useful when learning from
postedits where contrastive markings indicate human error corrections to the
original hypotheses. Code is publicly released.