Mitigating Negative Transfer with Task Awareness for Sexism, Hate
  Speech, and Toxic Language Detection
Angel Felipe Magnossão de Paula Paolo Rosso Damiano Spina
This paper proposes a novelty approach to mitigate the negative transfer
problem. In the field of machine learning, the common strategy is to apply the
Single-Task Learning approach in order to train a supervised model to solve a
specific task. Training a robust model requires a lot of data and a significant
amount of computational resources, making this solution unfeasible in cases
where data are unavailable or expensive to gather. Therefore another solution,
based on the sharing of information between tasks, has been developed:
Multi-Task Learning (MTL). Despite the recent developments regarding MTL, the
problem of negative transfer has still to be solved. Negative transfer is a
phenomenon that occurs when noisy information is shared between tasks,
resulting in a drop in performance. This paper proposes a new approach to
mitigate the negative transfer problem based on the task awareness concept. The
proposed approach results in diminishing the negative transfer together with an
improvement of performance over classic MTL solution. Moreover, the proposed
approach has been implemented in two unified architectures to detect Sexism,
Hate Speech, and Toxic Language in text comments. The proposed architectures
set a new state-of-the-art both in EXIST-2021 and HatEval-2019 benchmarks.