Householder Projector for Unsupervised Latent Semantics Discovery
Yue Song Jichao Zhang Nicu Sebe Wei Wang
Generative Adversarial Networks (GANs), especially the recent style-based
generators (StyleGANs), have versatile semantics in the structured latent
space. Latent semantics discovery methods emerge to move around the latent code
such that only one factor varies during the traversal. Recently, an
unsupervised method proposed a promising direction to directly use the
eigenvectors of the projection matrix that maps latent codes to features as the
interpretable directions. However, one overlooked fact is that the projection
matrix is non-orthogonal and the number of eigenvectors is too large. The
non-orthogonality would entangle semantic attributes in the top few
eigenvectors, and the large dimensionality might result in meaningless
variations among the directions even if the matrix is orthogonal. To avoid
these issues, we propose Householder Projector, a flexible and general low-rank
orthogonal matrix representation based on Householder transformations, to
parameterize the projection matrix. The orthogonality guarantees that the
eigenvectors correspond to disentangled interpretable semantics, while the
low-rank property encourages that each identified direction has meaningful
variations. We integrate our projector into pre-trained StyleGAN2/StyleGAN3 and
evaluate the models on several benchmarks. Within only $1\%$ of the original
training steps for fine-tuning, our projector helps StyleGANs to discover more
disentangled and precise semantic attributes without sacrificing image
fidelity.