Navigating the Ocean of Biases: Political Bias Attribution in Language
  Models via Causal Structures
David F. Jenny Yann Billeter Mrinmaya Sachan Bernhard Schölkopf Zhijing Jin
The rapid advancement of Large Language Models (LLMs) has sparked intense
debate regarding their ability to perceive and interpret complex
socio-political landscapes. In this study, we undertake an exploration of
decision-making processes and inherent biases within LLMs, exemplified by
ChatGPT, specifically contextualizing our analysis within political debates. We
aim not to critique or validate LLMs' values, but rather to discern how they
interpret and adjudicate "good arguments." By applying Activity Dependency
Networks (ADNs), we extract the LLMs' implicit criteria for such assessments
and illustrate how normative values influence these perceptions. We discuss the
consequences of our findings for human-AI alignment and bias mitigation. Our
code and data at https://github.com/david-jenny/LLM-Political-Study.