Reacting like Humans: Incorporating Intrinsic Human Behaviors into NAO
  through Sound-Based Reactions for Enhanced Sociability
Ali Ghadami Mohammadreza Taghimohammadi Mohammad Mohammadzadeh Mohammad Hosseinipour Alireza Taheri
Robots' acceptability among humans and their sociability can be significantly
enhanced by incorporating human-like reactions. Humans can react to
environmental events very quickly and without thinking. An instance where
humans display natural reactions is when they encounter a sudden and loud sound
that startles or frightens them. During such moments, individuals may
instinctively move their hands, turn toward the origin of the sound, and try to
determine the event's cause. This inherent behavior motivated us to explore
this less-studied part of social robotics. In this work, a multi-modal system
composed of an action generator, sound classifier, and YOLO object detector was
designed to sense the environment and, in the presence of sudden loud sounds,
show natural human fear reactions, and finally, locate the fear-causing sound
source in the environment. These unique and valid generated motions and
inferences could imitate intrinsic human reactions and enhance the sociability
of robots. For motion generation, a model based on LSTM and MDN networks was
proposed to synthesize various motions. Also, in the case of sound detection, a
transfer learning model was preferred that used the spectrogram of sound
signals as its input. After developing individual models for sound detection,
motion generation, and image recognition, they were integrated into a
comprehensive fear module that was implemented on the NAO robot. Finally, the
fear module was tested in practical application and two groups of experts and
non-experts filled out a questionnaire to evaluate the performance of the
robot. Given our promising results, this preliminary exploratory research
provides a fresh perspective on social robotics and could be a starting point
for modeling intrinsic human behaviors and emotions in robots.