Implicit Design Choices and Their Impact on Emotion Recognition Model
  Development and Evaluation
Mimansa Jaiswal
Emotion recognition is a complex task due to the inherent subjectivity in
both the perception and production of emotions. The subjectivity of emotions
poses significant challenges in developing accurate and robust computational
models. This thesis examines critical facets of emotion recognition, beginning
with the collection of diverse datasets that account for psychological factors
in emotion production.
  To handle the challenge of non-representative training data, this work
collects the Multimodal Stressed Emotion dataset, which introduces controlled
stressors during data collection to better represent real-world influences on
emotion production. To address issues with label subjectivity, this research
comprehensively analyzes how data augmentation techniques and annotation
schemes impact emotion perception and annotator labels. It further handles
natural confounding variables and variations by employing adversarial networks
to isolate key factors like stress from learned emotion representations during
model training. For tackling concerns about leakage of sensitive demographic
variables, this work leverages adversarial learning to strip sensitive
demographic information from multimodal encodings. Additionally, it proposes
optimized sociological evaluation metrics aligned with cost-effective,
real-world needs for model testing.
  This research advances robust, practical emotion recognition through
multifaceted studies of challenges in datasets, labels, modeling, demographic
and membership variable encoding in representations, and evaluation. The
groundwork has been laid for cost-effective, generalizable emotion recognition
models that are less likely to encode sensitive demographic information.