The Persuasive Power of Large Language Models
Simon Martin Breum Daniel Vædele Egdal Victor Gram Mortensen Anders Giovanni Møller Luca Maria Aiello
The increasing capability of Large Language Models to act as human-like
social agents raises two important questions in the area of opinion dynamics.
First, whether these agents can generate effective arguments that could be
injected into the online discourse to steer the public opinion. Second, whether
artificial agents can interact with each other to reproduce dynamics of
persuasion typical of human social systems, opening up opportunities for
studying synthetic social systems as faithful proxies for opinion dynamics in
human populations. To address these questions, we designed a synthetic
persuasion dialogue scenario on the topic of climate change, where a
'convincer' agent generates a persuasive argument for a 'skeptic' agent, who
subsequently assesses whether the argument changed its internal opinion state.
Different types of arguments were generated to incorporate different linguistic
dimensions underpinning psycho-linguistic theories of opinion change. We then
asked human judges to evaluate the persuasiveness of machine-generated
arguments. Arguments that included factual knowledge, markers of trust,
expressions of support, and conveyed status were deemed most effective
according to both humans and agents, with humans reporting a marked preference
for knowledge-based arguments. Our experimental framework lays the groundwork
for future in-silico studies of opinion dynamics, and our findings suggest that
artificial agents have the potential of playing an important role in collective
processes of opinion formation in online social media.