Crossing Linguistic Horizons: Finetuning and Comprehensive Evaluation of
  Vietnamese Large Language Models
Sang T. Truong Duc Q. Nguyen Toan Nguyen Dong D. Le Nhi N. Truong Tho Quan Sanmi Koyejo
Recent advancements in large language models (LLMs) have underscored their
importance in the evolution of artificial intelligence. However, despite
extensive pretraining on multilingual datasets, available open-sourced LLMs
exhibit limited effectiveness in processing Vietnamese. The challenge is
exacerbated by the absence of systematic benchmark datasets and metrics
tailored for Vietnamese LLM evaluation. To mitigate these issues, we have
finetuned LLMs specifically for Vietnamese and developed a comprehensive
evaluation framework encompassing 10 common tasks and 31 metrics. Our
evaluation results reveal that the fine-tuned LLMs exhibit enhanced
comprehension and generative capabilities in Vietnamese. Moreover, our analysis
indicates that models with more parameters can introduce more biases and
uncalibrated outputs and the key factor influencing LLM performance is the
quality of the training or fine-tuning datasets. These insights underscore the
significance of meticulous fine-tuning with high-quality datasets in enhancing
LLM performance.