A Wide Evaluation of ChatGPT on Affective Computing Tasks
Mostafa M. Amin Rui Mao Erik Cambria Björn W. Schuller
With the rise of foundation models, a new artificial intelligence paradigm
has emerged, by simply using general purpose foundation models with prompting
to solve problems instead of training a separate machine learning model for
each problem. Such models have been shown to have emergent properties of
solving problems that they were not initially trained on. The studies for the
effectiveness of such models are still quite limited. In this work, we widely
study the capabilities of the ChatGPT models, namely GPT-4 and GPT-3.5, on 13
affective computing problems, namely aspect extraction, aspect polarity
classification, opinion extraction, sentiment analysis, sentiment intensity
ranking, emotions intensity ranking, suicide tendency detection, toxicity
detection, well-being assessment, engagement measurement, personality
assessment, sarcasm detection, and subjectivity detection. We introduce a
framework to evaluate the ChatGPT models on regression-based problems, such as
intensity ranking problems, by modelling them as pairwise ranking
classification. We compare ChatGPT against more traditional NLP methods, such
as end-to-end recurrent neural networks and transformers. The results
demonstrate the emergent abilities of the ChatGPT models on a wide range of
affective computing problems, where GPT-3.5 and especially GPT-4 have shown
strong performance on many problems, particularly the ones related to
sentiment, emotions, or toxicity. The ChatGPT models fell short for problems
with implicit signals, such as engagement measurement and subjectivity
detection.