Octopus: A Multitask Model and Toolkit for Arabic Natural Language
  Generation
AbdelRahim Elmadany El Moatez Billah Nagoudi Muhammad Abdul-Mageed
Understanding Arabic text and generating human-like responses is a
challenging endeavor. While many researchers have proposed models and solutions
for individual problems, there is an acute shortage of a comprehensive Arabic
natural language generation toolkit that is capable of handling a wide range of
tasks. In this work, we present a novel Arabic text-to-text Transformer model,
namely AraT5v2. Our new model is methodically trained on extensive and diverse
data, utilizing an extended sequence length of 2,048 tokens. We explore various
pretraining strategies including unsupervised, supervised, and joint
pertaining, under both single and multitask settings. Our models outperform
competitive baselines with large margins. We take our work one step further by
developing and publicly releasing Octopus, a Python-based package and
command-line toolkit tailored for eight Arabic generation tasks all exploiting
a single model. We release the models and the toolkit on our public repository.