TURNA: A Turkish Encoder-Decoder Language Model for Enhanced
  Understanding and Generation
Gökçe Uludoğan Zeynep Yirmibeşoğlu Balal Furkan Akkurt Melikşah Türker Onur Güngör Susan Üsküdarlı
The recent advances in natural language processing have predominantly favored
well-resourced English-centric models, resulting in a significant gap with
low-resource languages. In this work, we introduce the language model TURNA,
which is developed for the low-resource language Turkish and is capable of both
natural language understanding and generation tasks. TURNA is pretrained with
an encoder-decoder architecture based on the unified framework UL2 with a
diverse corpus that we specifically curated for this purpose. We evaluated
TURNA with three generation tasks and five understanding tasks for Turkish. The
results show that TURNA outperforms several multilingual models in both
understanding and generation tasks, and competes with monolingual Turkish
models in understanding tasks. TURNA is made available at
https://huggingface.co/boun-tabi-LMG/TURNA .