Large Language Model Adaptation for Financial Sentiment Analysis
Pau Rodriguez Inserte Mariam Nakhlé Raheel Qader Gaetan Caillaut Jingshu Liu
Natural language processing (NLP) has recently gained relevance within
financial institutions by providing highly valuable insights into companies and
markets' financial documents. However, the landscape of the financial domain
presents extra challenges for NLP, due to the complexity of the texts and the
use of specific terminology. Generalist language models tend to fall short in
tasks specifically tailored for finance, even when using large language models
(LLMs) with great natural language understanding and generative capabilities.
This paper presents a study on LLM adaptation methods targeted at the financial
domain and with high emphasis on financial sentiment analysis. To this purpose,
two foundation models with less than 1.5B parameters have been adapted using a
wide range of strategies. We show that through careful fine-tuning on both
financial documents and instructions, these foundation models can be adapted to
the target domain. Moreover, we observe that small LLMs have comparable
performance to larger scale models, while being more efficient in terms of
parameters and data. In addition to the models, we show how to generate
artificial instructions through LLMs to augment the number of samples of the
instruction dataset.