Adaptive Domain Inference Attack
Yuechun Gu Keke Chen
As deep neural networks are increasingly deployed in sensitive application
domains, such as healthcare and security, it's necessary to understand what
kind of sensitive information can be inferred from these models. Existing
model-targeted attacks all assume the attacker has known the application domain
or training data distribution, which plays an essential role in successful
attacks. Can removing the domain information from model APIs protect models
from these attacks? This paper studies this critical problem. Unfortunately,
even with minimal knowledge, i.e., accessing the model as an unnamed function
without leaking the meaning of input and output, the proposed adaptive domain
inference attack (ADI) can still successfully estimate relevant subsets of
training data. We show that the extracted relevant data can significantly
improve, for instance, the performance of model-inversion attacks.
Specifically, the ADI method utilizes a concept hierarchy built on top of a
large collection of available public and private datasets and a novel algorithm
to adaptively tune the likelihood of leaf concepts showing up in the unseen
training data. The ADI attack not only extracts partial training data at the
concept level, but also converges fast and requires much fewer target-model
accesses than another domain inference attack, GDI.