Multilingual Controllable Transformer-Based Lexical Simplification
Kim Cheng Sheang Horacio Saggion
Text is by far the most ubiquitous source of knowledge and information and
should be made easily accessible to as many people as possible; however, texts
often contain complex words that hinder reading comprehension and
accessibility. Therefore, suggesting simpler alternatives for complex words
without compromising meaning would help convey the information to a broader
audience. This paper proposes mTLS, a multilingual controllable
Transformer-based Lexical Simplification (LS) system fined-tuned with the T5
model. The novelty of this work lies in the use of language-specific prefixes,
control tokens, and candidates extracted from pre-trained masked language
models to learn simpler alternatives for complex words. The evaluation results
on three well-known LS datasets -- LexMTurk, BenchLS, and NNSEval -- show that
our model outperforms the previous state-of-the-art models like LSBert and
ConLS. Moreover, further evaluation of our approach on the part of the recent
TSAR-2022 multilingual LS shared-task dataset shows that our model performs
competitively when compared with the participating systems for English LS and
even outperforms the GPT-3 model on several metrics. Moreover, our model
obtains performance gains also for Spanish and Portuguese.