FairBranch: Fairness Conflict Correction on Task-group Branches for Fair
  Multi-Task Learning
Arjun Roy Christos Koutlis Symeon Papadopoulos Eirini Ntoutsi
The generalization capacity of Multi-Task Learning (MTL) becomes limited when
unrelated tasks negatively impact each other by updating shared parameters with
conflicting gradients, resulting in negative transfer and a reduction in MTL
accuracy compared to single-task learning (STL). Recently, there has been an
increasing focus on the fairness of MTL models, necessitating the optimization
of both accuracy and fairness for individual tasks. Similarly to how negative
transfer affects accuracy, task-specific fairness considerations can adversely
influence the fairness of other tasks when there is a conflict of fairness loss
gradients among jointly learned tasks, termed bias transfer. To address both
negative and bias transfer in MTL, we introduce a novel method called
FairBranch. FairBranch branches the MTL model by assessing the similarity of
learned parameters, grouping related tasks to mitigate negative transfer.
Additionally, it incorporates fairness loss gradient conflict correction
between adjoining task-group branches to address bias transfer within these
task groups. Our experiments in tabular and visual MTL problems demonstrate
that FairBranch surpasses state-of-the-art MTL methods in terms of both
fairness and accuracy.