Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection
Erik Arakelyan Arnav Arora Isabelle Augenstein
Stance Detection is concerned with identifying the attitudes expressed by an
author towards a target of interest. This task spans a variety of domains
ranging from social media opinion identification to detecting the stance for a
legal claim. However, the framing of the task varies within these domains, in
terms of the data collection protocol, the label dictionary and the number of
available annotations. Furthermore, these stance annotations are significantly
imbalanced on a per-topic and inter-topic basis. These make multi-domain stance
detection a challenging task, requiring standardization and domain adaptation.
To overcome this challenge, we propose $\textbf{T}$opic $\textbf{E}$fficient
$\textbf{St}$anc$\textbf{E}$ $\textbf{D}$etection (TESTED), consisting of a
topic-guided diversity sampling technique and a contrastive objective that is
used for fine-tuning a stance classifier. We evaluate the method on an existing
benchmark of $16$ datasets with in-domain, i.e. all topics seen and
out-of-domain, i.e. unseen topics, experiments. The results show that our
method outperforms the state-of-the-art with an average of $3.5$ F1 points
increase in-domain, and is more generalizable with an averaged increase of
$10.2$ F1 on out-of-domain evaluation while using $\leq10\%$ of the training
data. We show that our sampling technique mitigates both inter- and per-topic
class imbalances. Finally, our analysis demonstrates that the contrastive
learning objective allows the model a more pronounced segmentation of samples
with varying labels.