Rating Prediction in Conversational Task Assistants with Behavioral and
  Conversational-Flow Features
Rafael Ferreira David Semedo João Magalhães
Predicting the success of Conversational Task Assistants (CTA) can be
critical to understand user behavior and act accordingly. In this paper, we
propose TB-Rater, a Transformer model which combines conversational-flow
features with user behavior features for predicting user ratings in a CTA
scenario. In particular, we use real human-agent conversations and ratings
collected in the Alexa TaskBot challenge, a novel multimodal and multi-turn
conversational context. Our results show the advantages of modeling both the
conversational-flow and behavioral aspects of the conversation in a single
model for offline rating prediction. Additionally, an analysis of the
CTA-specific behavioral features brings insights into this setting and can be
used to bootstrap future systems.