Towards Unbalanced Motion: Part-Decoupling Network for Video Portrait
  Segmentation
Tianshu Yu Changqun Xia Jia Li
Video portrait segmentation (VPS), aiming at segmenting prominent foreground
portraits from video frames, has received much attention in recent years.
However, simplicity of existing VPS datasets leads to a limitation on extensive
research of the task. In this work, we propose a new intricate large-scale
Multi-scene Video Portrait Segmentation dataset MVPS consisting of 101 video
clips in 7 scenario categories, in which 10,843 sampled frames are finely
annotated at pixel level. The dataset has diverse scenes and complicated
background environments, which is the most complex dataset in VPS to our best
knowledge. Through the observation of a large number of videos with portraits
during dataset construction, we find that due to the joint structure of human
body, motion of portraits is part-associated, which leads that different parts
are relatively independent in motion. That is, motion of different parts of the
portraits is unbalanced. Towards this unbalance, an intuitive and reasonable
idea is that different motion states in portraits can be better exploited by
decoupling the portraits into parts. To achieve this, we propose a
Part-Decoupling Network (PDNet) for video portrait segmentation. Specifically,
an Inter-frame Part-Discriminated Attention (IPDA) module is proposed which
unsupervisely segments portrait into parts and utilizes different attentiveness
on discriminative features specified to each different part. In this way,
appropriate attention can be imposed to portrait parts with unbalanced motion
to extract part-discriminated correlations, so that the portraits can be
segmented more accurately. Experimental results demonstrate that our method
achieves leading performance with the comparison to state-of-the-art methods.