Filter bubbles and affective polarization in user-personalized large
  language model outputs
Tomo Lazovich
Echoing the history of search engines and social media content rankings, the
advent of large language models (LLMs) has led to a push for increased
personalization of model outputs to individual users. In the past, personalized
recommendations and ranking systems have been linked to the development of
filter bubbles (serving content that may confirm a user's existing biases) and
affective polarization (strong negative sentiment towards those with differing
views). In this work, we explore how prompting a leading large language model,
ChatGPT-3.5, with a user's political affiliation prior to asking factual
questions about public figures and organizations leads to differing results. We
observe that left-leaning users tend to receive more positive statements about
left-leaning political figures and media outlets, while right-leaning users see
more positive statements about right-leaning entities. This pattern holds
across presidential candidates, members of the U.S. Senate, and media
organizations with ratings from AllSides. When qualitatively evaluating some of
these outputs, there is evidence that particular facts are included or excluded
based on the user's political affiliation. These results illustrate that
personalizing LLMs based on user demographics carry the same risks of affective
polarization and filter bubbles that have been seen in other personalized
internet technologies. This ``failure mode" should be monitored closely as
there are more attempts to monetize and personalize these models.