DiffuSIA: A Spiral Interaction Architecture for Encoder-Decoder Text
  Diffusion
Chao-Hong Tan Jia-Chen Gu Zhen-Hua Ling
Diffusion models have emerged as the new state-of-the-art family of deep
generative models, and their promising potentials for text generation have
recently attracted increasing attention. Existing studies mostly adopt a single
encoder architecture with partially noising processes for conditional text
generation, but its degree of flexibility for conditional modeling is limited.
In fact, the encoder-decoder architecture is naturally more flexible for its
detachable encoder and decoder modules, which is extensible to multilingual and
multimodal generation tasks for conditions and target texts. However, the
encoding process of conditional texts lacks the understanding of target texts.
To this end, a spiral interaction architecture for encoder-decoder text
diffusion (DiffuSIA) is proposed. Concretely, the conditional information from
encoder is designed to be captured by the diffusion decoder, while the target
information from decoder is designed to be captured by the conditional encoder.
These two types of information flow run through multilayer interaction spirally
for deep fusion and understanding. DiffuSIA is evaluated on four text
generation tasks, including paraphrase, text simplification, question
generation, and open-domain dialogue generation. Experimental results show that
DiffuSIA achieves competitive performance among previous methods on all four
tasks, demonstrating the effectiveness and generalization ability of the
proposed method.