Meta-Auxiliary Network for 3D GAN Inversion
Bangrui Jiang Zhenhua Guo Yujiu Yang
Real-world image manipulation has achieved fantastic progress in recent
years. GAN inversion, which aims to map the real image to the latent code
faithfully, is the first step in this pipeline. However, existing GAN inversion
methods fail to achieve high reconstruction quality and fast inference at the
same time. In addition, existing methods are built on 2D GANs and lack
explicitly mechanisms to enforce multi-view consistency.In this work, we
present a novel meta-auxiliary framework, while leveraging the newly developed
3D GANs as generator. The proposed method adopts a two-stage strategy. In the
first stage, we invert the input image to an editable latent code using
off-the-shelf inversion techniques. The auxiliary network is proposed to refine
the generator parameters with the given image as input, which both predicts
offsets for weights of convolutional layers and sampling positions of volume
rendering. In the second stage, we perform meta-learning to fast adapt the
auxiliary network to the input image, then the final reconstructed image is
synthesized via the meta-learned auxiliary network. Extensive experiments show
that our method achieves better performances on both inversion and editing
tasks.