Towards Effective Ancient Chinese Translation: Dataset, Model, and
  Evaluation
Geyang Guo Jiarong Yang Fengyuan Lu Jiaxin Qin Tianyi Tang Wayne Xin Zhao
Interpreting ancient Chinese has been the key to comprehending vast Chinese
literature, tradition, and civilization. In this paper, we propose Erya for
ancient Chinese translation. From a dataset perspective, we collect, clean, and
classify ancient Chinese materials from various sources, forming the most
extensive ancient Chinese resource to date. From a model perspective, we devise
Erya training method oriented towards ancient Chinese. We design two
jointly-working tasks: disyllabic aligned substitution (DAS) and dual masked
language model (DMLM). From an evaluation perspective, we build a benchmark to
judge ancient Chinese translation quality in different scenarios and evaluate
the ancient Chinese translation capacities of various existing models. Our
model exhibits remarkable zero-shot performance across five domains, with over
+12.0 BLEU against GPT-3.5 models and better human evaluation results than
ERNIE Bot. Subsequent fine-tuning further shows the superior transfer
capability of Erya model with +6.2 BLEU gain. We release all the
above-mentioned resources at https://github.com/RUCAIBox/Erya.