Evaluating Subjective Cognitive Appraisals of Emotions from Large
  Language Models
Hongli Zhan Desmond C. Ong Junyi Jessy Li
The emotions we experience involve complex processes; besides physiological
aspects, research in psychology has studied cognitive appraisals where people
assess their situations subjectively, according to their own values (Scherer,
2005). Thus, the same situation can often result in different emotional
experiences. While the detection of emotion is a well-established task, there
is very limited work so far on the automatic prediction of cognitive
appraisals. This work fills the gap by presenting CovidET-Appraisals, the most
comprehensive dataset to-date that assesses 24 appraisal dimensions, each with
a natural language rationale, across 241 Reddit posts. CovidET-Appraisals
presents an ideal testbed to evaluate the ability of large language models --
excelling at a wide range of NLP tasks -- to automatically assess and explain
cognitive appraisals. We found that while the best models are performant,
open-sourced LLMs fall short at this task, presenting a new challenge in the
future development of emotionally intelligent models. We release our dataset at
https://github.com/honglizhan/CovidET-Appraisals-Public.