Addressing the Length Bias Problem in Document-Level Neural Machine
  Translation
Zhuocheng Zhang Shuhao Gu Min Zhang Yang Feng
Document-level neural machine translation (DNMT) has shown promising results
by incorporating more context information. However, this approach also
introduces a length bias problem, whereby DNMT suffers from significant
translation quality degradation when decoding documents that are much shorter
or longer than the maximum sequence length during training. %i.e., the length
bias problem. To solve the length bias problem, we propose to improve the DNMT
model in training method, attention mechanism, and decoding strategy. Firstly,
we propose to sample the training data dynamically to ensure a more uniform
distribution across different sequence lengths. Then, we introduce a
length-normalized attention mechanism to aid the model in focusing on target
information, mitigating the issue of attention divergence when processing
longer sequences. Lastly, we propose a sliding window strategy during decoding
that integrates as much context information as possible without exceeding the
maximum sequence length. The experimental results indicate that our method can
bring significant improvements on several open datasets, and further analysis
shows that our method can significantly alleviate the length bias problem.