Unlocking the Emotional World of Visual Media: An Overview of the
  Science, Research, and Impact of Understanding Emotion
James Z. Wang Sicheng Zhao Chenyan Wu Reginald B. Adams Michelle G. Newman Tal Shafir Rachelle Tsachor
The emergence of artificial emotional intelligence technology is
revolutionizing the fields of computers and robotics, allowing for a new level
of communication and understanding of human behavior that was once thought
impossible. While recent advancements in deep learning have transformed the
field of computer vision, automated understanding of evoked or expressed
emotions in visual media remains in its infancy. This foundering stems from the
absence of a universally accepted definition of "emotion", coupled with the
inherently subjective nature of emotions and their intricate nuances. In this
article, we provide a comprehensive, multidisciplinary overview of the field of
emotion analysis in visual media, drawing on insights from psychology,
engineering, and the arts. We begin by exploring the psychological foundations
of emotion and the computational principles that underpin the understanding of
emotions from images and videos. We then review the latest research and systems
within the field, accentuating the most promising approaches. We also discuss
the current technological challenges and limitations of emotion analysis,
underscoring the necessity for continued investigation and innovation. We
contend that this represents a "Holy Grail" research problem in computing and
delineate pivotal directions for future inquiry. Finally, we examine the
ethical ramifications of emotion-understanding technologies and contemplate
their potential societal impacts. Overall, this article endeavors to equip
readers with a deeper understanding of the domain of emotion analysis in visual
media and to inspire further research and development in this captivating and
rapidly evolving field.