Are Large Language Models Rational Investors?
Yuhang Zhou Yuchen Ni Xiang Liu Jian Zhang Sen Liu Guangnan Ye Hongfeng Chai
Large Language Models (LLMs) are progressively being adopted in financial
analysis to harness their extensive knowledge base for interpreting complex
market data and trends. However, their application in the financial domain is
challenged by intrinsic biases (i.e., risk-preference bias) and a superficial
grasp of market intricacies, underscoring the need for a thorough assessment of
their financial insight. This study introduces a novel framework, Financial
Bias Indicators (FBI), to critically evaluate the financial rationality of
LLMs, focusing on their ability to discern and navigate the subtleties of
financial information and to identify any irrational biases that might skew
market analysis.
  Our research adopts an innovative methodology to measure financial
rationality, integrating principles of behavioral finance to scrutinize the
biases and decision-making patterns of LLMs. We conduct a comprehensive
evaluation of 19 leading LLMs, considering factors such as model scale,
training datasets, input strategies, etc. The findings reveal varying degrees
of financial irrationality among the models, influenced by their design and
training. Models trained specifically on financial datasets might exhibit
greater irrationality, and it's possible that even larger financial language
models (FinLLMs) could display more biases than smaller, more generalized
models. This outcomes provide profound insights into how these elements affect
the financial rationality of LLMs, indicating that targeted training and
structured input methods could improve model performance. This work enriches
our understanding of LLMs' strengths and weaknesses in financial applications,
laying the groundwork for the development of more dependable and rational
financial analysis tools.