Persistent-Transient Duality: A Multi-mechanism Approach for Modeling
  Human-Object Interaction
Hung Tran Vuong Le Svetha Venkatesh Truyen Tran
Humans are highly adaptable, swiftly switching between different modes to
progressively handle different tasks, situations and contexts. In Human-object
interaction (HOI) activities, these modes can be attributed to two mechanisms:
(1) the large-scale consistent plan for the whole activity and (2) the
small-scale children interactive actions that start and end along the timeline.
While neuroscience and cognitive science have confirmed this multi-mechanism
nature of human behavior, machine modeling approaches for human motion are
trailing behind. While attempted to use gradually morphing structures (e.g.,
graph attention networks) to model the dynamic HOI patterns, they miss the
expeditious and discrete mode-switching nature of the human motion. To bridge
that gap, this work proposes to model two concurrent mechanisms that jointly
control human motion: the Persistent process that runs continually on the
global scale, and the Transient sub-processes that operate intermittently on
the local context of the human while interacting with objects. These two
mechanisms form an interactive Persistent-Transient Duality that
synergistically governs the activity sequences. We model this conceptual
duality by a parent-child neural network of Persistent and Transient channels
with a dedicated neural module for dynamic mechanism switching. The framework
is trialed on HOI motion forecasting. On two rich datasets and a wide variety
of settings, the model consistently delivers superior performances, proving its
suitability for the challenge.