CrudeBERT: Applying Economic Theory towards fine-tuning
  Transformer-based Sentiment Analysis Models to the Crude Oil Market
Himmet Kaplan Ralf-Peter Mundani Heiko Rölke Albert Weichselbraun
Predicting market movements based on the sentiment of news media has a long
tradition in data analysis. With advances in natural language processing,
transformer architectures have emerged that enable contextually aware sentiment
classification. Nevertheless, current methods built for the general financial
market such as FinBERT cannot distinguish asset-specific value-driving factors.
This paper addresses this shortcoming by presenting a method that identifies
and classifies events that impact supply and demand in the crude oil markets
within a large corpus of relevant news headlines. We then introduce CrudeBERT,
a new sentiment analysis model that draws upon these events to contextualize
and fine-tune FinBERT, thereby yielding improved sentiment classifications for
headlines related to the crude oil futures market. An extensive evaluation
demonstrates that CrudeBERT outperforms proprietary and open-source solutions
in the domain of crude oil.