PersianMind: A Cross-Lingual Persian-English Large Language Model
Pedram Rostami Ali Salemi Mohammad Javad Dousti
Large language models demonstrate remarkable proficiency in various
linguistic tasks and have extensive knowledge across various domains. Although
they perform best in English, their ability in other languages is notable too.
In contrast, open-source models, such as LLaMa, are primarily trained on
English datasets, resulting in poor performance in non-English languages. In
this paper, we introduce PersianMind, an open-source bilingual large language
model which demonstrates comparable performance to closed-source GPT-3.5-turbo
in the Persian language. By expanding LLaMa2's vocabulary with 10,000 Persian
tokens and training it on a dataset comprising nearly 2 billion Persian tokens,
we show that our approach preserves the model's English knowledge and employs
transfer learning to excel at transferring task knowledge from one language to
another.