It's MBR All the Way Down: Modern Generation Techniques Through the Lens
  of Minimum Bayes Risk
Amanda Bertsch Alex Xie Graham Neubig Matthew R. Gormley
Minimum Bayes Risk (MBR) decoding is a method for choosing the outputs of a
machine learning system based not on the output with the highest probability,
but the output with the lowest risk (expected error) among multiple candidates.
It is a simple but powerful method: for an additional cost at inference time,
MBR provides reliable several-point improvements across metrics for a wide
variety of tasks without any additional data or training. Despite this, MBR is
not frequently applied in NLP works, and knowledge of the method itself is
limited. We first provide an introduction to the method and the recent
literature. We show that several recent methods that do not reference MBR can
be written as special cases of MBR; this reformulation provides additional
theoretical justification for the performance of these methods, explaining some
results that were previously only empirical. We provide theoretical and
empirical results about the effectiveness of various MBR variants and make
concrete recommendations for the application of MBR in NLP models, including
future directions in this area.