Non-autoregressive Text Editing with Copy-aware Latent Alignments
Yu Zhang Yue Zhang Leyang Cui Guohong Fu
Recent work has witnessed a paradigm shift from Seq2Seq to Seq2Edit in the
field of text editing, with the aim of addressing the slow autoregressive
inference problem posed by the former. Despite promising results, Seq2Edit
approaches still face several challenges such as inflexibility in generation
and difficulty in generalizing to other languages. In this work, we propose a
novel non-autoregressive text editing method to circumvent the above issues, by
modeling the edit process with latent CTC alignments. We make a crucial
extension to CTC by introducing the copy operation into the edit space, thus
enabling more efficient management of textual overlap in editing. We conduct
extensive experiments on GEC and sentence fusion tasks, showing that our
proposed method significantly outperforms existing Seq2Edit models and achieves
similar or even better results than Seq2Seq with over $4\times$ speedup.
Moreover, it demonstrates good generalizability on German and Russian. In-depth
analyses reveal the strengths of our method in terms of the robustness under
various scenarios and generating fluent and flexible outputs.