Unsupervised Accuracy Estimation of Deep Visual Models using
  Domain-Adaptive Adversarial Perturbation without Source Samples
JoonHo Lee Jae Oh Woo Hankyu Moon Kwonho Lee
Deploying deep visual models can lead to performance drops due to the
discrepancies between source and target distributions. Several approaches
leverage labeled source data to estimate target domain accuracy, but accessing
labeled source data is often prohibitively difficult due to data
confidentiality or resource limitations on serving devices. Our work proposes a
new framework to estimate model accuracy on unlabeled target data without
access to source data. We investigate the feasibility of using pseudo-labels
for accuracy estimation and evolve this idea into adopting recent advances in
source-free domain adaptation algorithms. Our approach measures the
disagreement rate between the source hypothesis and the target pseudo-labeling
function, adapted from the source hypothesis. We mitigate the impact of
erroneous pseudo-labels that may arise due to a high ideal joint hypothesis
risk by employing adaptive adversarial perturbation on the input of the target
model. Our proposed source-free framework effectively addresses the challenging
distribution shift scenarios and outperforms existing methods requiring source
data and labels for training.