Improving Dialogue Agents by Decomposing One Global Explicit Annotation
  with Local Implicit Multimodal Feedback
Dong Won Lee Hae Won Park Yoon Kim Cynthia Breazeal Louis-Philippe Morency
We describe an approach for aligning an LLM-based dialogue agent based on
global (i.e., dialogue-level) rewards, while also taking into account
naturally-occurring multimodal signals. At a high level, our approach (dubbed
GELI) learns a local, turn-level reward model by decomposing the human-provided
Global Explicit (GE) session-level reward, using Local Implicit (LI} multimodal
reward signals to crossmodally shape the reward decomposition step. This
decomposed reward model is then used as part of the standard RHLF pipeline
improve an LLM-based dialog agent. We run quantitative and qualitative human
studies to evaluate the performance of our GELI approach, and find that it
shows consistent improvements across various conversational metrics compared to
baseline methods.