Secure Aggregation is Not Private Against Membership Inference Attacks
Khac-Hoang Ngo Johan Östman Giuseppe Durisi Alexandre Graell i Amat
Secure aggregation (SecAgg) is a commonly-used privacy-enhancing mechanism in
federated learning, affording the server access only to the aggregate of model
updates while safeguarding the confidentiality of individual updates. Despite
widespread claims regarding SecAgg's privacy-preserving capabilities, a formal
analysis of its privacy is lacking, making such presumptions unjustified. In
this paper, we delve into the privacy implications of SecAgg by treating it as
a local differential privacy (LDP) mechanism for each local update. We design a
simple attack wherein an adversarial server seeks to discern which update
vector a client submitted, out of two possible ones, in a single training round
of federated learning under SecAgg. By conducting privacy auditing, we assess
the success probability of this attack and quantify the LDP guarantees provided
by SecAgg. Our numerical results unveil that, contrary to prevailing claims,
SecAgg offers weak privacy against membership inference attacks even in a
single training round. Indeed, it is difficult to hide a local update by adding
other independent local updates when the updates are of high dimension. Our
findings underscore the imperative for additional privacy-enhancing mechanisms,
such as noise injection, in federated learning.