Closing the Loop: Testing ChatGPT to Generate Model Explanations to
  Improve Human Labelling of Sponsored Content on Social Media
Thales Bertaglia Stefan Huber Catalina Goanta Gerasimos Spanakis Adriana Iamnitchi
Regulatory bodies worldwide are intensifying their efforts to ensure
transparency in influencer marketing on social media through instruments like
the Unfair Commercial Practices Directive (UCPD) in the European Union, or
Section 5 of the Federal Trade Commission Act. Yet enforcing these obligations
has proven to be highly problematic due to the sheer scale of the influencer
market. The task of automatically detecting sponsored content aims to enable
the monitoring and enforcement of such regulations at scale. Current research
in this field primarily frames this problem as a machine learning task,
focusing on developing models that achieve high classification performance in
detecting ads. These machine learning tasks rely on human data annotation to
provide ground truth information. However, agreement between annotators is
often low, leading to inconsistent labels that hinder the reliability of
models. To improve annotation accuracy and, thus, the detection of sponsored
content, we propose using chatGPT to augment the annotation process with
phrases identified as relevant features and brief explanations. Our experiments
show that this approach consistently improves inter-annotator agreement and
annotation accuracy. Additionally, our survey of user experience in the
annotation task indicates that the explanations improve the annotators'
confidence and streamline the process. Our proposed methods can ultimately lead
to more transparency and alignment with regulatory requirements in sponsored
content detection.