Don't Look into the Dark: Latent Codes for Pluralistic Image Inpainting
Haiwei Chen Yajie Zhao
We present a method for large-mask pluralistic image inpainting based on the
generative framework of discrete latent codes. Our method learns latent priors,
discretized as tokens, by only performing computations at the visible locations
of the image. This is realized by a restrictive partial encoder that predicts
the token label for each visible block, a bidirectional transformer that infers
the missing labels by only looking at these tokens, and a dedicated synthesis
network that couples the tokens with the partial image priors to generate
coherent and pluralistic complete image even under extreme mask settings.
Experiments on public benchmarks validate our design choices as the proposed
method outperforms strong baselines in both visual quality and diversity
metrics.