Fast Training of NMT Model with Data Sorting
Daniela N. Rim Kimera Richard Heeyoul Choi
The Transformer model has revolutionized Natural Language Processing tasks
such as Neural Machine Translation, and many efforts have been made to study
the Transformer architecture, which increased its efficiency and accuracy. One
potential area for improvement is to address the computation of empty tokens
that the Transformer computes only to discard them later, leading to an
unnecessary computational burden. To tackle this, we propose an algorithm that
sorts translation sentence pairs based on their length before batching,
minimizing the waste of computing power. Since the amount of sorting could
violate the independent and identically distributed (i.i.d) data assumption, we
sort the data partially. In experiments, we apply the proposed method to
English-Korean and English-Luganda language pairs for machine translation and
show that there are gains in computational time while maintaining the
performance. Our method is independent of architectures, so that it can be
easily integrated into any training process with flexible data lengths.