DIVERSE: Deciphering Internet Views on the U.S. Military Through Video
  Comment Stance Analysis, A Novel Benchmark Dataset for Stance Classification
Iain J. Cruickshank Lynnette Hui Xian Ng
Stance detection of social media text is a key component of downstream tasks
involving the identification of groups of users with opposing opinions on
contested topics such as vaccination and within arguments. In particular,
stance provides an indication of an opinion towards an entity. This paper
introduces DIVERSE, a dataset of over 173,000 YouTube video comments annotated
for their stance towards videos of the U.S. military. The stance is annotated
through a human-guided, machine-assisted labeling methodology that makes use of
weak signals of tone within the sentence as supporting indicators, as opposed
to using manual annotations by humans. These weak signals consist of the
presence of hate speech and sarcasm, the presence of specific keywords, the
sentiment of the text, and the stance inference from two Large Language Models.
The weak signals are then consolidated using a data programming model before
each comment is annotated with a final stance label. On average, the videos
have 200 comments each, and the stance of the comments skews slightly towards
the "against" characterization for both the U.S. Army and the videos posted on
the channel.