Domain Bridge: Generative model-based domain forensic for black-box
  models
Jiyi Zhang Han Fang Ee-Chien Chang
In forensic investigations of machine learning models, techniques that
determine a model's data domain play an essential role, with prior work relying
on large-scale corpora like ImageNet to approximate the target model's domain.
Although such methods are effective in finding broad domains, they often
struggle in identifying finer-grained classes within those domains. In this
paper, we introduce an enhanced approach to determine not just the general data
domain (e.g., human face) but also its specific attributes (e.g., wearing
glasses). Our approach uses an image embedding model as the encoder and a
generative model as the decoder. Beginning with a coarse-grained description,
the decoder generates a set of images, which are then presented to the unknown
target model. Successful classifications by the model guide the encoder to
refine the description, which in turn, are used to produce a more specific set
of images in the subsequent iteration. This iterative refinement narrows down
the exact class of interest. A key strength of our approach lies in leveraging
the expansive dataset, LAION-5B, on which the generative model Stable Diffusion
is trained. This enlarges our search space beyond traditional corpora, such as
ImageNet. Empirical results showcase our method's performance in identifying
specific attributes of a model's input domain, paving the way for more detailed
forensic analyses of deep learning models.