MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki
Timothee Mickus Stig-Arne Grönroos Joseph Attieh Michele Boggia Ona De Gibert Shaoxiong Ji Niki Andreas Lopi Alessandro Raganato Raúl Vázquez Jörg Tiedemann
NLP in the age of monolithic large language models is approaching its limits
in terms of size and information that can be handled. The trend goes to
modularization, a necessary step into the direction of designing smaller
sub-networks and components with specialized functionality. In this paper, we
present the MAMMOTH toolkit: a framework designed for training massively
multilingual modular machine translation systems at scale, initially derived
from OpenNMT-py and then adapted to ensure efficient training across
computation clusters. We showcase its efficiency across clusters of A100 and
V100 NVIDIA GPUs, and discuss our design philosophy and plans for future
information. The toolkit is publicly available online.