SentAlign: Accurate and Scalable Sentence Alignment
Steinþór Steingrímsson Hrafn Loftsson Andy Way
We present SentAlign, an accurate sentence alignment tool designed to handle
very large parallel document pairs. Given user-defined parameters, the
alignment algorithm evaluates all possible alignment paths in fairly large
documents of thousands of sentences and uses a divide-and-conquer approach to
align documents containing tens of thousands of sentences. The scoring function
is based on LaBSE bilingual sentence representations. SentAlign outperforms
five other sentence alignment tools when evaluated on two different evaluation
sets, German-French and English-Icelandic, and on a downstream machine
translation task.