IMTLab: An Open-Source Platform for Building, Evaluating, and Diagnosing
  Interactive Machine Translation Systems
Xu Huang Zhirui Zhang Ruize Gao Yichao Du Lemao Liu Gouping Huang Shuming Shi Jiajun Chen Shujian Huang
We present IMTLab, an open-source end-to-end interactive machine translation
(IMT) system platform that enables researchers to quickly build IMT systems
with state-of-the-art models, perform an end-to-end evaluation, and diagnose
the weakness of systems. IMTLab treats the whole interactive translation
process as a task-oriented dialogue with a human-in-the-loop setting, in which
human interventions can be explicitly incorporated to produce high-quality,
error-free translations. To this end, a general communication interface is
designed to support the flexible IMT architectures and user policies. Based on
the proposed design, we construct a simulated and real interactive environment
to achieve end-to-end evaluation and leverage the framework to systematically
evaluate previous IMT systems. Our simulated and manual experiments show that
the prefix-constrained decoding approach still gains the lowest editing cost in
the end-to-end evaluation, while BiTIIMT achieves comparable editing cost with
a better interactive experience.