When a Language Question Is at Stake. A Revisited Approach to Label
  Sensitive Content
Stetsenko Daria
Many under-resourced languages require high-quality datasets for specific
tasks such as offensive language detection, disinformation, or misinformation
identification. However, the intricacies of the content may have a detrimental
effect on the annotators. The article aims to revisit an approach of
pseudo-labeling sensitive data on the example of Ukrainian tweets covering the
Russian-Ukrainian war. Nowadays, this acute topic is in the spotlight of
various language manipulations that cause numerous disinformation and profanity
on social media platforms. The conducted experiment highlights three main
stages of data annotation and underlines the main obstacles during machine
annotation. Ultimately, we provide a fundamental statistical analysis of the
obtained data, evaluation of models used for pseudo-labelling, and set further
guidelines on how the scientists can leverage the corpus to execute more
advanced research and extend the existing data samples without annotators'
engagement.