Attacking Perceptual Similarity Metrics
Abhijay Ghildyal Feng Liu
Perceptual similarity metrics have progressively become more correlated with
human judgments on perceptual similarity; however, despite recent advances, the
addition of an imperceptible distortion can still compromise these metrics. In
our study, we systematically examine the robustness of these metrics to
imperceptible adversarial perturbations. Following the two-alternative
forced-choice experimental design with two distorted images and one reference
image, we perturb the distorted image closer to the reference via an
adversarial attack until the metric flips its judgment. We first show that all
metrics in our study are susceptible to perturbations generated via common
adversarial attacks such as FGSM, PGD, and the One-pixel attack. Next, we
attack the widely adopted LPIPS metric using spatial-transformation-based
adversarial perturbations (stAdv) in a white-box setting to craft adversarial
examples that can effectively transfer to other similarity metrics in a
black-box setting. We also combine the spatial attack stAdv with PGD
($\ell_\infty$-bounded) attack to increase transferability and use these
adversarial examples to benchmark the robustness of both traditional and
recently developed metrics. Our benchmark provides a good starting point for
discussion and further research on the robustness of metrics to imperceptible
adversarial perturbations.