Towards Collaborative Plan Acquisition through Theory of Mind Modeling
  in Situated Dialogue
Cristian-Paul Bara Ziqiao Ma Yingzhuo Yu Julie Shah Joyce Chai
Collaborative tasks often begin with partial task knowledge and incomplete
initial plans from each partner. To complete these tasks, agents need to engage
in situated communication with their partners and coordinate their partial
plans towards a complete plan to achieve a joint task goal. While such
collaboration seems effortless in a human-human team, it is highly challenging
for human-AI collaboration. To address this limitation, this paper takes a step
towards collaborative plan acquisition, where humans and agents strive to learn
and communicate with each other to acquire a complete plan for joint tasks.
Specifically, we formulate a novel problem for agents to predict the missing
task knowledge for themselves and for their partners based on rich perceptual
and dialogue history. We extend a situated dialogue benchmark for symmetric
collaborative tasks in a 3D blocks world and investigate computational
strategies for plan acquisition. Our empirical results suggest that predicting
the partner's missing knowledge is a more viable approach than predicting one's
own. We show that explicit modeling of the partner's dialogue moves and mental
states produces improved and more stable results than without. These results
provide insight for future AI agents that can predict what knowledge their
partner is missing and, therefore, can proactively communicate such information
to help their partner acquire such missing knowledge toward a common
understanding of joint tasks.