Stance Detection on Social Media with Fine-Tuned Large Language Models
İlker Gül Rémi Lebret Karl Aberer
Stance detection, a key task in natural language processing, determines an
author's viewpoint based on textual analysis. This study evaluates the
evolution of stance detection methods, transitioning from early machine
learning approaches to the groundbreaking BERT model, and eventually to modern
Large Language Models (LLMs) such as ChatGPT, LLaMa-2, and Mistral-7B. While
ChatGPT's closed-source nature and associated costs present challenges, the
open-source models like LLaMa-2 and Mistral-7B offers an encouraging
alternative. Initially, our research focused on fine-tuning ChatGPT, LLaMa-2,
and Mistral-7B using several publicly available datasets. Subsequently, to
provide a comprehensive comparison, we assess the performance of these models
in zero-shot and few-shot learning scenarios. The results underscore the
exceptional ability of LLMs in accurately detecting stance, with all tested
models surpassing existing benchmarks. Notably, LLaMa-2 and Mistral-7B
demonstrate remarkable efficiency and potential for stance detection, despite
their smaller sizes compared to ChatGPT. This study emphasizes the potential of
LLMs in stance detection and calls for more extensive research in this field.