Tamil-Llama: A New Tamil Language Model Based on Llama 2
Abhinand Balachandran
Language modeling has witnessed remarkable advancements in recent years, with
Large Language Models (LLMs) like ChatGPT setting unparalleled benchmarks in
human-like text generation. However, a prevailing limitation is the
underrepresentation of languages like Tamil in these cutting-edge models,
leading to suboptimal performance in diverse linguistic contexts. This paper
addresses this lacuna, enhancing the open-source LLaMA model with an addition
of 16,000 Tamil tokens, aiming to achieve superior text generation and
comprehension in the Tamil language. We strategically employ the LoRA
methodology for efficient model training on a comprehensive Tamil corpus,
ensuring computational feasibility and model robustness. Moreover, we introduce
a Tamil-translated version of the Alpaca dataset and a subset of the OpenOrca
dataset tailored for instruction fine-tuning. Our results showcase significant
performance improvements in Tamil text generation, with potential implications
for the broader landscape of LLMs in Indian languages. We further underscore
our commitment to open research by making our models, datasets, and code
publicly accessible, fostering further innovations in language modeling.