A Simple Framework to Accelerate Multilingual Language Model for
  Monolingual Text Generation
Jimin Hong Gibbeum Lee Jaewoong Cho
Recent advancements in large language models have facilitated the execution
of complex language tasks, not only in English but also in non-English
languages. However, the tokenizers of most language models, such as Llama,
trained on English-centric corpora, tend to excessively fragment tokens in
non-English languages. This issue is especially pronounced in non-roman
alphabetic languages, which are often divided at a character or even Unicode
level, leading to slower text generation. To address this, our study introduces
a novel framework designed to expedite text generation in these languages. This
framework predicts larger linguistic units than those of conventional
multilingual tokenizers and is specifically tailored to the target language,
thereby reducing the number of decoding steps required. Our empirical results
demonstrate that the proposed framework increases the generation speed by a
factor of 1.9 compared to standard decoding while maintaining the performance
of a pre-trained multilingual model on monolingual tasks.