Investigating Large Language Models' Perception of Emotion Using
  Appraisal Theory
Nutchanon Yongsatianchot Parisa Ghanad Torshizi Stacy Marsella
Large Language Models (LLM) like ChatGPT have significantly advanced in
recent years and are now being used by the general public. As more people
interact with these systems, improving our understanding of these black box
models is crucial, especially regarding their understanding of human
psychological aspects. In this work, we investigate their emotion perception
through the lens of appraisal and coping theory using the Stress and Coping
Process Questionaire (SCPQ). SCPQ is a validated clinical instrument consisting
of multiple stories that evolve over time and differ in key appraisal variables
such as controllability and changeability. We applied SCPQ to three recent LLMs
from OpenAI, davinci-003, ChatGPT, and GPT-4 and compared the results with
predictions from the appraisal theory and human data. The results show that
LLMs' responses are similar to humans in terms of dynamics of appraisal and
coping, but their responses did not differ along key appraisal dimensions as
predicted by the theory and data. The magnitude of their responses is also
quite different from humans in several variables. We also found that GPTs can
be quite sensitive to instruction and how questions are asked. This work adds
to the growing literature evaluating the psychological aspects of LLMs and
helps enrich our understanding of the current models.