Reference-Based 3D-Aware Image Editing with Triplane
Bahri Batuhan Bilecen Yigit Yalin Ning Yu Aysegul Dundar
Generative Adversarial Networks (GANs) have emerged as powerful tools not
only for high-quality image generation but also for real image editing through
manipulation of their interpretable latent spaces. Recent advancements in GANs
include the development of 3D-aware models such as EG3D, characterized by
efficient triplane-based architectures enabling the reconstruction of 3D
geometry from single images. However, scant attention has been devoted to
providing an integrated framework for high-quality reference-based 3D-aware
image editing within this domain. This study addresses this gap by exploring
and demonstrating the effectiveness of EG3D's triplane space for achieving
advanced reference-based edits, presenting a unique perspective on 3D-aware
image editing through our novel pipeline. Our approach integrates the encoding
of triplane features, spatial disentanglement and automatic localization of
features in the triplane domain, and fusion learning for desired image editing.
Moreover, our framework demonstrates versatility across domains, extending its
effectiveness to animal face edits and partial stylization of cartoon
portraits. The method shows significant improvements over relevant 3D-aware
latent editing and 2D reference-based editing methods, both qualitatively and
quantitatively. Project page: https://three-bee.github.io/triplane_edit