Improving Natural Language Inference in Arabic using Transformer Models
  and Linguistically Informed Pre-Training
Mohammad Majd Saad Al Deen Maren Pielka Jörn Hees Bouthaina Soulef Abdou Rafet Sifa
This paper addresses the classification of Arabic text data in the field of
Natural Language Processing (NLP), with a particular focus on Natural Language
Inference (NLI) and Contradiction Detection (CD). Arabic is considered a
resource-poor language, meaning that there are few data sets available, which
leads to limited availability of NLP methods. To overcome this limitation, we
create a dedicated data set from publicly available resources. Subsequently,
transformer-based machine learning models are being trained and evaluated. We
find that a language-specific model (AraBERT) performs competitively with
state-of-the-art multilingual approaches, when we apply linguistically informed
pre-training methods such as Named Entity Recognition (NER). To our knowledge,
this is the first large-scale evaluation for this task in Arabic, as well as
the first application of multi-task pre-training in this context.