Arc2Face: A Foundation Model of Human Faces
Foivos Paraperas Papantoniou Alexandros Lattas Stylianos Moschoglou Jiankang Deng Bernhard Kainz Stefanos Zafeiriou
This paper presents Arc2Face, an identity-conditioned face foundation model,
which, given the ArcFace embedding of a person, can generate diverse
photo-realistic images with an unparalleled degree of face similarity than
existing models. Despite previous attempts to decode face recognition features
into detailed images, we find that common high-resolution datasets (e.g. FFHQ)
lack sufficient identities to reconstruct any subject. To that end, we
meticulously upsample a significant portion of the WebFace42M database, the
largest public dataset for face recognition (FR). Arc2Face builds upon a
pretrained Stable Diffusion model, yet adapts it to the task of ID-to-face
generation, conditioned solely on ID vectors. Deviating from recent works that
combine ID with text embeddings for zero-shot personalization of text-to-image
models, we emphasize on the compactness of FR features, which can fully capture
the essence of the human face, as opposed to hand-crafted prompts. Crucially,
text-augmented models struggle to decouple identity and text, usually
necessitating some description of the given face to achieve satisfactory
similarity. Arc2Face, however, only needs the discriminative features of
ArcFace to guide the generation, offering a robust prior for a plethora of
tasks where ID consistency is of paramount importance. As an example, we train
a FR model on synthetic images from our model and achieve superior performance
to existing synthetic datasets.