Virtual Accessory Try-On via Keypoint Hallucination
Junhong Gou Bo Zhang Li Niu Jianfu Zhang Jianlou Si Chen Qian Liqing Zhang
The virtual try-on task refers to fitting the clothes from one image onto
another portrait image. In this paper, we focus on virtual accessory try-on,
which fits accessory (e.g., glasses, ties) onto a face or portrait image.
Unlike clothing try-on, which relies on human silhouette as guidance, accessory
try-on warps the accessory into an appropriate location and shape to generate a
plausible composite image. In contrast to previous try-on methods that treat
foreground (i.e., accessories) and background (i.e., human faces or bodies)
equally, we propose a background-oriented network to utilize the prior
knowledge of human bodies and accessories. Specifically, our approach learns
the human body priors and hallucinates the target locations of specified
foreground keypoints in the background. Then our approach will inject
foreground information with accessory priors into the background UNet. Based on
the hallucinated target locations, the warping parameters are calculated to
warp the foreground. Moreover, this background-oriented network can also easily
incorporate auxiliary human face/body semantic segmentation supervision to
further boost performance. Experiments conducted on STRAT dataset validate the
effectiveness of our proposed method.