TransGOP: Transformer-Based Gaze Object Prediction
Binglu Wang Chenxi Guo Yang Jin Haisheng Xia Nian Liu
Gaze object prediction aims to predict the location and category of the
object that is watched by a human. Previous gaze object prediction works use
CNN-based object detectors to predict the object's location. However, we find
that Transformer-based object detectors can predict more accurate object
location for dense objects in retail scenarios. Moreover, the long-distance
modeling capability of the Transformer can help to build relationships between
the human head and the gaze object, which is important for the GOP task. To
this end, this paper introduces Transformer into the fields of gaze object
prediction and proposes an end-to-end Transformer-based gaze object prediction
method named TransGOP. Specifically, TransGOP uses an off-the-shelf
Transformer-based object detector to detect the location of objects and designs
a Transformer-based gaze autoencoder in the gaze regressor to establish
long-distance gaze relationships. Moreover, to improve gaze heatmap regression,
we propose an object-to-gaze cross-attention mechanism to let the queries of
the gaze autoencoder learn the global-memory position knowledge from the object
detector. Finally, to make the whole framework end-to-end trained, we propose a
Gaze Box loss to jointly optimize the object detector and gaze regressor by
enhancing the gaze heatmap energy in the box of the gaze object. Extensive
experiments on the GOO-Synth and GOO-Real datasets demonstrate that our
TransGOP achieves state-of-the-art performance on all tracks, i.e., object
detection, gaze estimation, and gaze object prediction. Our code will be
available at https://github.com/chenxi-Guo/TransGOP.git.