Non-autoregressive Machine Translation with Probabilistic Context-free
  Grammar
Shangtong Gui Chenze Shao Zhengrui Ma Xishan Zhang Yunji Chen Yang Feng
Non-autoregressive Transformer(NAT) significantly accelerates the inference
of neural machine translation. However, conventional NAT models suffer from
limited expression power and performance degradation compared to autoregressive
(AT) models due to the assumption of conditional independence among target
tokens. To address these limitations, we propose a novel approach called
PCFG-NAT, which leverages a specially designed Probabilistic Context-Free
Grammar (PCFG) to enhance the ability of NAT models to capture complex
dependencies among output tokens. Experimental results on major machine
translation benchmarks demonstrate that PCFG-NAT further narrows the gap in
translation quality between NAT and AT models. Moreover, PCFG-NAT facilitates a
deeper understanding of the generated sentences, addressing the lack of
satisfactory explainability in neural machine translation.Code is publicly
available at https://github.com/ictnlp/PCFG-NAT.