GreedLlama: Performance of Financial Value-Aligned Large Language Models
  in Moral Reasoning
Jeffy Yu Maximilian Huber Kevin Tang
This paper investigates the ethical implications of aligning Large Language
Models (LLMs) with financial optimization, through the case study of
GreedLlama, a model fine-tuned to prioritize economically beneficial outcomes.
By comparing GreedLlama's performance in moral reasoning tasks to a base Llama2
model, our results highlight a concerning trend: GreedLlama demonstrates a
marked preference for profit over ethical considerations, making morally
appropriate decisions at significantly lower rates than the base model in
scenarios of both low and high moral ambiguity. In low ambiguity situations,
GreedLlama's ethical decisions decreased to 54.4%, compared to the base model's
86.9%, while in high ambiguity contexts, the rate was 47.4% against the base
model's 65.1%. These findings emphasize the risks of single-dimensional value
alignment in LLMs, underscoring the need for integrating broader ethical values
into AI development to ensure decisions are not solely driven by financial
incentives. The study calls for a balanced approach to LLM deployment,
advocating for the incorporation of ethical considerations in models intended
for business applications, particularly in light of the absence of regulatory
oversight.