Enhancing Zero-Shot Crypto Sentiment with Fine-tuned Language Model and
  Prompt Engineering
Rahman S M Wahidur Ishmam Tashdeed Manjit Kaur Heung-No-Lee
Blockchain technology has revolutionized the financial landscape, with
cryptocurrencies gaining widespread adoption for their decentralized and
transparent nature. As the sentiment expressed on social media platforms can
significantly influence cryptocurrency discussions and market movements,
sentiment analysis has emerged as a crucial tool for understanding public
opinion and predicting market trends. Motivated by the aim to enhance sentiment
analysis accuracy in the cryptocurrency domain, this paper investigates
fine-tuning techniques on large language models. This paper also investigates
the efficacy of supervised fine-tuning and instruction-based fine-tuning on
large language models for unseen tasks. Experimental results demonstrate a
significant average zero-shot performance gain of 40% after fine-tuning,
highlighting the potential of this technique in optimizing pre-trained language
model efficiency. Additionally, the impact of instruction tuning on models of
varying scales is examined, revealing that larger models benefit from
instruction tuning, achieving the highest average accuracy score of 75.16%. In
contrast, smaller-scale models may experience reduced generalization due to the
complete utilization of model capacity. To gain deeper insight about how
instruction works with these language models, this paper presents an
experimental investigation into the response of an instruction-based model
under different instruction tuning setups. The investigation demonstrates that
the model achieves an average accuracy score of 72.38% for short and simple
instructions. This performance significantly outperforms its accuracy under
long and complex instructions by over 12%, thereby effectively highlighting the
profound significance of instruction characteristics in maximizing model
performance.