GC-VTON: Predicting Globally Consistent and Occlusion Aware Local Flows
  with Neighborhood Integrity Preservation for Virtual Try-on
Hamza Rawal Muhammad Junaid Ahmad Farooq Zaman
Flow based garment warping is an integral part of image-based virtual try-on
networks. However, optimizing a single flow predicting network for simultaneous
global boundary alignment and local texture preservation results in sub-optimal
flow fields. Moreover, dense flows are inherently not suited to handle
intricate conditions like garment occlusion by body parts or by other garments.
Forcing flows to handle the above issues results in various distortions like
texture squeezing, and stretching. In this work, we propose a novel approach
where we disentangle the global boundary alignment and local texture preserving
tasks via our GlobalNet and LocalNet modules. A consistency loss is then
employed between the two modules which harmonizes the local flows with the
global boundary alignment. Additionally, we explicitly handle occlusions by
predicting body-parts visibility mask, which is used to mask out the occluded
regions in the warped garment. The masking prevents the LocalNet from
predicting flows that distort texture to compensate for occlusions. We also
introduce a novel regularization loss (NIPR), that defines a criteria to
identify the regions in the warped garment where texture integrity is violated
(squeezed or stretched). NIPR subsequently penalizes the flow in those regions
to ensure regular and coherent warps that preserve the texture in local
neighborhoods. Evaluation on a widely used virtual try-on dataset demonstrates
strong performance of our network compared to the current SOTA methods.