AMII: Adaptive Multimodal Inter-personal and Intra-personal Model for
  Adapted Behavior Synthesis
Jieyeon Woo Mireille Fares Catherine Pelachaud Catherine Achard
Socially Interactive Agents (SIAs) are physical or virtual embodied agents
that display similar behavior as human multimodal behavior. Modeling SIAs'
non-verbal behavior, such as speech and facial gestures, has always been a
challenging task, given that a SIA can take the role of a speaker or a
listener. A SIA must emit appropriate behavior adapted to its own speech, its
previous behaviors (intra-personal), and the User's behaviors (inter-personal)
for both roles. We propose AMII, a novel approach to synthesize adaptive facial
gestures for SIAs while interacting with Users and acting interchangeably as a
speaker or as a listener. AMII is characterized by modality memory encoding
schema - where modality corresponds to either speech or facial gestures - and
makes use of attention mechanisms to capture the intra-personal and
inter-personal relationships. We validate our approach by conducting objective
evaluations and comparing it with the state-of-the-art approaches.