Neural Conversation Models and How to Rein Them in: A Survey of Failures
  and Fixes
Fabian Galetzka Anne Beyer David Schlangen
Recent conditional language models are able to continue any kind of text
source in an often seemingly fluent way. This fact encouraged research in the
area of open-domain conversational systems that are based on powerful language
models and aim to imitate an interlocutor by generating appropriate
contributions to a written dialogue. From a linguistic perspective, however,
the complexity of contributing to a conversation is high. In this survey, we
interpret Grice's maxims of cooperative conversation from the perspective of
this specific research area and systematize the literature under the aspect of
what makes a contribution appropriate: A neural conversation model has to be
fluent, informative, consistent, coherent, and follow social norms. In order to
ensure these qualities, recent approaches try to tame the underlying language
models at various intervention points, such as data, training regime or
decoding. Sorted by these categories and intervention points, we discuss
promising attempts and suggest novel ways for future research.