Disentangled Interaction Representation for One-Stage Human-Object
  Interaction Detection
Xubin Zhong Changxing Ding Yupeng Hu Dacheng Tao
Human-Object Interaction (HOI) detection is a core task for human-centric
image understanding. Recent one-stage methods adopt a transformer decoder to
collect image-wide cues that are useful for interaction prediction; however,
the interaction representations obtained using this method are entangled and
lack interpretability. In contrast, traditional two-stage methods benefit
significantly from their ability to compose interaction features in a
disentangled and explainable manner. In this paper, we improve the performance
of one-stage methods by enabling them to extract disentangled interaction
representations. First, we propose Shunted Cross-Attention (SCA) to extract
human appearance, object appearance, and global context features using
different cross-attention heads. This is achieved by imposing different masks
on the cross-attention maps produced by the different heads. Second, we
introduce the Interaction-aware Pose Estimation (IPE) task to learn
interaction-relevant human pose features using a disentangled decoder. This is
achieved with a novel attention module that accurately captures the human
keypoints relevant to the current interaction category. Finally, our approach
fuses the appearance feature and pose feature via element-wise addition to form
the interaction representation. Experimental results show that our approach can
be readily applied to existing one-stage HOI detectors. Moreover, we achieve
state-of-the-art performance on two benchmarks: HICO-DET and V-COCO.