A Large Collection of Model-generated Contradictory Responses for
  Consistency-aware Dialogue Systems
Shiki Sato Reina Akama Jun Suzuki Kentaro Inui
Mitigating the generation of contradictory responses poses a substantial
challenge in dialogue response generation. The quality and quantity of
available contradictory response data play a vital role in suppressing these
contradictions, offering two significant benefits. First, having access to
large contradiction data enables a comprehensive examination of their
characteristics. Second, data-driven methods to mitigate contradictions may be
enhanced with large-scale contradiction data for training. Nevertheless, no
attempt has been made to build an extensive collection of model-generated
contradictory responses. In this paper, we build a large dataset of response
generation models' contradictions for the first time. Then, we acquire valuable
insights into the characteristics of model-generated contradictions through an
extensive analysis of the collected responses. Lastly, we also demonstrate how
this dataset substantially enhances the performance of data-driven
contradiction suppression methods.