Cognitive Bias in High-Stakes Decision-Making with LLMs
Jessica Echterhoff Yao Liu Abeer Alessa Julian McAuley Zexue He
Large language models (LLMs) offer significant potential as tools to support
an expanding range of decision-making tasks. However, given their training on
human (created) data, LLMs can inherit both societal biases against protected
groups, as well as be subject to cognitive bias. Such human-like bias can
impede fair and explainable decisions made with LLM assistance. Our work
introduces BiasBuster, a framework designed to uncover, evaluate, and mitigate
cognitive bias in LLMs, particularly in high-stakes decision-making tasks.
Inspired by prior research in psychology and cognitive sciences, we develop a
dataset containing 16,800 prompts to evaluate different cognitive biases (e.g.,
prompt-induced, sequential, inherent). We test various bias mitigation
strategies, amidst proposing a novel method using LLMs to debias their own
prompts. Our analysis provides a comprehensive picture on the presence and
effects of cognitive bias across different commercial and open-source models.
We demonstrate that our self-help debiasing effectively mitigate cognitive bias
without having to manually craft examples for each bias type.