Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4
  on mock CFA Exams
Ethan Callanan Amarachi Mbakwe Antony Papadimitriou Yulong Pei Mathieu Sibue Xiaodan Zhu Zhiqiang Ma Xiaomo Liu Sameena Shah
Large Language Models (LLMs) have demonstrated remarkable performance on a
wide range of Natural Language Processing (NLP) tasks, often matching or even
beating state-of-the-art task-specific models. This study aims at assessing the
financial reasoning capabilities of LLMs. We leverage mock exam questions of
the Chartered Financial Analyst (CFA) Program to conduct a comprehensive
evaluation of ChatGPT and GPT-4 in financial analysis, considering Zero-Shot
(ZS), Chain-of-Thought (CoT), and Few-Shot (FS) scenarios. We present an
in-depth analysis of the models' performance and limitations, and estimate
whether they would have a chance at passing the CFA exams. Finally, we outline
insights into potential strategies and improvements to enhance the
applicability of LLMs in finance. In this perspective, we hope this work paves
the way for future studies to continue enhancing LLMs for financial reasoning
through rigorous evaluation.