Deep Privacy Funnel Model: From a Discriminative to a Generative
  Approach with an Application to Face Recognition
Behrooz Razeghi Parsa Rahimi Sébastien Marcel
In this study, we apply the information-theoretic Privacy Funnel (PF) model
to the domain of face recognition, developing a novel method for
privacy-preserving representation learning within an end-to-end training
framework. Our approach addresses the trade-off between obfuscation and utility
in data protection, quantified through logarithmic loss, also known as
self-information loss. This research provides a foundational exploration into
the integration of information-theoretic privacy principles with representation
learning, focusing specifically on the face recognition systems. We
particularly highlight the adaptability of our framework with recent
advancements in face recognition networks, such as AdaFace and ArcFace. In
addition, we introduce the Generative Privacy Funnel ($\mathsf{GenPF}$) model,
a paradigm that extends beyond the traditional scope of the PF model, referred
to as the Discriminative Privacy Funnel ($\mathsf{DisPF}$). This
$\mathsf{GenPF}$ model brings new perspectives on data generation methods with
estimation-theoretic and information-theoretic privacy guarantees.
Complementing these developments, we also present the deep variational PF
(DVPF) model. This model proposes a tractable variational bound for measuring
information leakage, enhancing the understanding of privacy preservation
challenges in deep representation learning. The DVPF model, associated with
both $\mathsf{DisPF}$ and $\mathsf{GenPF}$ models, sheds light on connections
with various generative models such as Variational Autoencoders (VAEs),
Generative Adversarial Networks (GANs), and Diffusion models. Complementing our
theoretical contributions, we release a reproducible PyTorch package,
facilitating further exploration and application of these privacy-preserving
methodologies in face recognition systems.