Injecting knowledge into language generation: a case study in
  auto-charting after-visit care instructions from medical dialogue
Maksim Eremeev Ilya Valmianski Xavier Amatriain Anitha Kannan
Factual correctness is often the limiting factor in practical applications of
natural language generation in high-stakes domains such as healthcare. An
essential requirement for maintaining factuality is the ability to deal with
rare tokens. This paper focuses on rare tokens that appear in both the source
and the reference sequences, and which, when missed during generation, decrease
the factual correctness of the output text. For high-stake domains that are
also knowledge-rich, we show how to use knowledge to (a) identify which rare
tokens that appear in both source and reference are important and (b) uplift
their conditional probability. We introduce the ``utilization rate'' that
encodes knowledge and serves as a regularizer by maximizing the marginal
probability of selected tokens. We present a study in a knowledge-rich domain
of healthcare, where we tackle the problem of generating after-visit care
instructions based on patient-doctor dialogues. We verify that, in our dataset,
specific medical concepts with high utilization rates are underestimated by
conventionally trained sequence-to-sequence models. We observe that correcting
this with our approach to knowledge injection reduces the uncertainty of the
model as well as improves factuality and coherence without negatively impacting
fluency.