Detecting Generated Native Ads in Conversational Search
Sebastian Schmidt Ines Zelch Janek Bevendorff Benno Stein Matthias Hagen Martin Potthast
Conversational search engines such as YouChat and Microsoft Copilot use large
language models (LLMs) to generate answers to queries. It is only a small step
to also use this technology to generate and integrate advertising within these
answers - instead of placing ads separately from the organic search results.
This type of advertising is reminiscent of native advertising and product
placement, both of which are very effective forms of subtle and manipulative
advertising. It is likely that information seekers will be confronted with such
use of LLM technology in the near future, especially when considering the high
computational costs associated with LLMs, for which providers need to develop
sustainable business models. This paper investigates whether LLMs can also be
used as a countermeasure against generated native ads, i.e., to block them. For
this purpose we compile a large dataset of ad-prone queries and of generated
answers with automatically integrated ads to experiment with fine-tuned
sentence transformers and state-of-the-art LLMs on the task of recognizing the
ads. In our experiments sentence transformers achieve detection precision and
recall values above 0.9, while the investigated LLMs struggle with the task.