Zero-shot Text-driven Physically Interpretable Face Editing
Yapeng Meng Songru Yang Xu Hu Rui Zhao Lincheng Li Zhenwei Shi Zhengxia Zou
This paper proposes a novel and physically interpretable method for face
editing based on arbitrary text prompts. Different from previous
GAN-inversion-based face editing methods that manipulate the latent space of
GANs, or diffusion-based methods that model image manipulation as a reverse
diffusion process, we regard the face editing process as imposing vector flow
fields on face images, representing the offset of spatial coordinates and color
for each image pixel. Under the above-proposed paradigm, we represent the
vector flow field in two ways: 1) explicitly represent the flow vectors with
rasterized tensors, and 2) implicitly parameterize the flow vectors as
continuous, smooth, and resolution-agnostic neural fields, by leveraging the
recent advances of implicit neural representations. The flow vectors are
iteratively optimized under the guidance of the pre-trained Contrastive
Language-Image Pretraining~(CLIP) model by maximizing the correlation between
the edited image and the text prompt. We also propose a learning-based one-shot
face editing framework, which is fast and adaptable to any text prompt input.
Our method can also be flexibly extended to real-time video face editing.
Compared with state-of-the-art text-driven face editing methods, our method can
generate physically interpretable face editing results with high identity
consistency and image quality. Our code will be made publicly available.