Rethinking Privacy in Machine Learning Pipelines from an Information
  Flow Control Perspective
Lukas Wutschitz Boris Köpf Andrew Paverd Saravan Rajmohan Ahmed Salem Shruti Tople Santiago Zanella-Béguelin Menglin Xia Victor Rühle
Modern machine learning systems use models trained on ever-growing corpora.
Typically, metadata such as ownership, access control, or licensing information
is ignored during training. Instead, to mitigate privacy risks, we rely on
generic techniques such as dataset sanitization and differentially private
model training, with inherent privacy/utility trade-offs that hurt model
performance. Moreover, these techniques have limitations in scenarios where
sensitive information is shared across multiple participants and fine-grained
access control is required. By ignoring metadata, we therefore miss an
opportunity to better address security, privacy, and confidentiality
challenges. In this paper, we take an information flow control perspective to
describe machine learning systems, which allows us to leverage metadata such as
access control policies and define clear-cut privacy and confidentiality
guarantees with interpretable information flows. Under this perspective, we
contrast two different approaches to achieve user-level non-interference: 1)
fine-tuning per-user models, and 2) retrieval augmented models that access
user-specific datasets at inference time. We compare these two approaches to a
trivially non-interfering zero-shot baseline using a public model and to a
baseline that fine-tunes this model on the whole corpus. We evaluate trained
models on two datasets of scientific articles and demonstrate that retrieval
augmented architectures deliver the best utility, scalability, and flexibility
while satisfying strict non-interference guarantees.