DiffusionDialog: A Diffusion Model for Diverse Dialog Generation with
  Latent Space
Jianxiang Xiang Zhenhua Liu Haodong Liu Yin Bai Jia Cheng Wenliang Chen
In real-life conversations, the content is diverse, and there exists the
one-to-many problem that requires diverse generation. Previous studies
attempted to introduce discrete or Gaussian-based continuous latent variables
to address the one-to-many problem, but the diversity is limited. Recently,
diffusion models have made breakthroughs in computer vision, and some attempts
have been made in natural language processing. In this paper, we propose
DiffusionDialog, a novel approach to enhance the diversity of dialogue
generation with the help of diffusion model. In our approach, we introduce
continuous latent variables into the diffusion model. The problem of using
latent variables in the dialog task is how to build both an effective prior of
the latent space and an inferring process to obtain the proper latent given the
context. By combining the encoder and latent-based diffusion model, we encode
the response's latent representation in a continuous space as the prior,
instead of fixed Gaussian distribution or simply discrete ones. We then infer
the latent by denoising step by step with the diffusion model. The experimental
results show that our model greatly enhances the diversity of dialog responses
while maintaining coherence. Furthermore, in further analysis, we find that our
diffusion model achieves high inference efficiency, which is the main challenge
of applying diffusion models in natural language processing.