Holistic Prototype Attention Network for Few-Shot VOS
Yin Tang Tao Chen Xiruo Jiang Yazhou Yao Guo-Sen Xie Heng-Tao Shen
Few-shot video object segmentation (FSVOS) aims to segment dynamic objects of
unseen classes by resorting to a small set of support images that contain
pixel-level object annotations. Existing methods have demonstrated that the
domain agent-based attention mechanism is effective in FSVOS by learning the
correlation between support images and query frames. However, the agent frame
contains redundant pixel information and background noise, resulting in
inferior segmentation performance. Moreover, existing methods tend to ignore
inter-frame correlations in query videos. To alleviate the above dilemma, we
propose a holistic prototype attention network (HPAN) for advancing FSVOS.
Specifically, HPAN introduces a prototype graph attention module (PGAM) and a
bidirectional prototype attention module (BPAM), transferring informative
knowledge from seen to unseen classes. PGAM generates local prototypes from all
foreground features and then utilizes their internal correlations to enhance
the representation of the holistic prototypes. BPAM exploits the holistic
information from support images and video frames by fusing co-attention and
self-attention to achieve support-query semantic consistency and inner-frame
temporal consistency. Extensive experiments on YouTube-FSVOS have been provided
to demonstrate the effectiveness and superiority of our proposed HPAN method.