Medical mT5: An Open-Source Multilingual Text-to-Text LLM for The
  Medical Domain
Iker García-Ferrero Rodrigo Agerri Aitziber Atutxa Salazar Elena Cabrio Iker de la Iglesia Alberto Lavelli Bernardo Magnini Benjamin Molinet Johana Ramirez-Romero German Rigau Jose Maria Villa-Gonzalez Serena Villata Andrea Zaninello
Research on language technology for the development of medical applications
is currently a hot topic in Natural Language Understanding and Generation.
Thus, a number of large language models (LLMs) have recently been adapted to
the medical domain, so that they can be used as a tool for mediating in
human-AI interaction. While these LLMs display competitive performance on
automated medical texts benchmarks, they have been pre-trained and evaluated
with a focus on a single language (English mostly). This is particularly true
of text-to-text models, which typically require large amounts of
domain-specific pre-training data, often not easily accessible for many
languages. In this paper, we address these shortcomings by compiling, to the
best of our knowledge, the largest multilingual corpus for the medical domain
in four languages, namely English, French, Italian and Spanish. This new corpus
has been used to train Medical mT5, the first open-source text-to-text
multilingual model for the medical domain. Additionally, we present two new
evaluation benchmarks for all four languages with the aim of facilitating
multilingual research in this domain. A comprehensive evaluation shows that
Medical mT5 outperforms both encoders and similarly sized text-to-text models
for the Spanish, French, and Italian benchmarks, while being competitive with
current state-of-the-art LLMs in English.