Utilizing Speech Emotion Recognition and Recommender Systems for
  Negative Emotion Handling in Therapy Chatbots
Farideh Majidi Marzieh Bahrami
Emotional well-being significantly influences mental health and overall
quality of life. As therapy chatbots become increasingly prevalent, their
ability to comprehend and respond empathetically to users' emotions remains
limited. This paper addresses this limitation by proposing an approach to
enhance therapy chatbots with auditory perception, enabling them to understand
users' feelings and provide human-like empathy. The proposed method
incorporates speech emotion recognition (SER) techniques using Convolutional
Neural Network (CNN) models and the ShEMO dataset to accurately detect and
classify negative emotions, including anger, fear, and sadness. The SER model
achieves a validation accuracy of 88%, demonstrating its effectiveness in
recognizing emotional states from speech signals. Furthermore, a recommender
system is developed, leveraging the SER model's output to generate personalized
recommendations for managing negative emotions, for which a new bilingual
dataset was generated as well since there is no such dataset available for this
task. The recommender model achieves an accuracy of 98% by employing a
combination of global vectors for word representation (GloVe) and LSTM models.
To provide a more immersive and empathetic user experience, a text-to-speech
model called GlowTTS is integrated, enabling the therapy chatbot to audibly
communicate the generated recommendations to users in both English and Persian.
The proposed approach offers promising potential to enhance therapy chatbots by
providing them with the ability to recognize and respond to users' emotions,
ultimately improving the delivery of mental health support for both English and
Persian-speaking users.