Deep neural networks are susceptible to adversarial attacks. In computer vision, well-crafted perturbations to images can cause neural networks to make mistakes such as confusing a cat with a computer. Previous adversarial attacks have been designed to degrade performance of models or cause machine learning models to produce specific outputs chosen ahead of time by the attacker. We introduce attacks that instead reprogram the target model to perform a task chosen by the attacker without the attacker needing to specify or compute the desired output for each test-time input. This attack finds a single adversarial perturbation, that can be added to all test-time inputs to a machine learning model in order to cause the model to perform a task chosen by the adversary—even if the model was not trained to do this task. These perturbations can thus be considered a program for the new task. We demonstrate adversarial reprogramming on six ImageNet classification models, repurposing these models to perform a counting task, as well as classification tasks: classification of MNIST and CIFAR-10 examples presented as inputs to the ImageNet model. The study of adversarial examples is often motivated in terms of the danger posed by an attacker whose goal is to cause model prediction errors with a small change to the model's input. Such an attacker could make a self-driving car react to a phantom stop sign BID6 by means of a sticker (a small L 0 perturbation), or cause an insurance company's damage model to overestimate the claim value from the resulting accident by subtly doctoring photos of the damage (a small L ∞ perturbation). With this context, various methods have been proposed both to construct BID40 BID32 BID3 BID24 and defend against BID9 BID17 BID25 BID42 BID14 BID13 this style of adversarial attack. Thus far, the majority of adversarial attacks have consisted of untargeted attacks that aim to degrade the performance of a model without necessarily requiring it to produce a specific output, or targeted attacks in which the attacker designs an adversarial perturbation to produce a specific output for that input. For example, an attack against a classifier might target a specific desired output class for each input image, or an attack against a reinforcement learning agent might induce that agent to enter a specific state BID22 .In practice, there is no requirement that adversarial attacks will adhere to this framework. Thus , it is crucial to proactively anticipate other unexplored adversarial goals in order to make machine learning systems more secure. In this work, we consider a novel and more challenging adversarial goal: reprogramming the model to perform a task chosen by the attacker, without the attacker needing to compute the specific desired output. Consider a model trained to perform some original task: for inputs x it produces outputs f (x). Consider an adversary who wishes to perform an adversarial task:for inputsx (not necessarily in the same domain as x) the adversary wishes to compute a function g(x). We show that an adversary can accomplish this by learning adversarial reprogramming functions h f (·; θ) and h g (·; θ) that map between the two tasks. Here, h f converts inputs from the domain of x into the domain of x (i.e., h f (x; θ) is a valid input to the function f ), while h g maps output of f (h(x; θ)) back to outputs of g (x) . The parameters θ of the adversarial program are then adjusted to achieve h g (f (h f (x))) = g (x).In our work, for simplicity, we definex to be a small image, g a function that processes small images, x a large image, and f a function that processes large images. Our function h f then just consists of drawing x in the center of the large image and θ in the borders (though see Section 4.5 for other schemes), and h g is simply a hard coded mapping between output class labels. However, the idea is more general; h f (h g ) could be any consistent transformation that converts between the input (output) formats for the two tasks and causes the model to perform the adversarial task.We refer to the class of attacks where a model is repurposed to perform a new task as adversarial reprogramming. We refer to θ as an adversarial program. In contrast to most previous adversarial work, the attack does not need to be imperceptible to humans, or even subtle, in order to be considered a success. However, we note that it is still possible to construct reprogramming attacks that are imperceptible. Potential consequences of adversarial reprogramming include theft of computational resources from public facing services, repurposing of AI-driven assistants into spies or spam bots, and abusing machine learning services for tasks violating the ethical principles of system providers. Risks stemming from this type of attack are discussed in more detail in Section 5.2.It may seem unlikely that an additive offset to a neural network's input would be sufficient on its own to repurpose the network to a new task. However, this flexibility stemming only from changes to a network's inputs is consistent with results on the expressive power of deep neural networks. For instance, in BID36 it is shown that, depending on network hyperparameters, the number of unique output patterns achievable by moving along a one-dimensional trajectory in input space increases exponentially with network depth. Further, BID21 shows that networks can often be trained to high accuracy even if parameter updates are restricted to occur only in a low dimensional subspace. An additive offset to a neural network's input is equivalent to a modification of its first layer biases (for a convolutional network with biases shared across space, this operation effectively introduces new parameters because the additive input is not shared across space), and therefore an adversarial program corresponds to an update in a low dimensional parameter subspace.In this paper, we present the first instances of adversarial reprogramming. In Section 2, we discuss related work. In Section 3, we present a training procedure for crafting adversarial programs, which cause a neural network to perform a new task. In Section 4, we experimentally demonstrate adversarial programs that target several convolutional neural networks designed to classify ImageNet data. These adversarial programs alter the network function from ImageNet classification to: counting squares in an image, classifying MNIST digits, and classifying CIFAR-10 images. Next, we examine the susceptibility of trained and untrained networks to adversarial reprogramming. We then demonstrate the possibility of reprograming adversarial tasks with adversarial data that has no resemblance to original data, demonstrating that results from transfer learning do not fully explain adversarial reprogramming. Further, we demonstrate the possibility of concealing adversarial programs and data. Finally, we end in Sections 5 and 6 by discussing and summarizing our results. In this work, we proposed a new class of adversarial attacks that aim to reprogram neural networks to perform novel adversarial tasks. Our results demonstrate for the first time the possibility of such attacks. They are also illustrative of both surprising flexibility and surprising vulnerability in deep neural networks. Future investigation should address the properties and limitations of adversarial reprogramming, and possible ways to mitigate or defend against it.
We introduce the first instance of adversarial attacks that reprogram the target model to perform a task chosen by the attacker---without the attacker needing to specify or compute the desired output for each test-time input. The authors present a novel adversarial attack scheme where a neural net is repurposed to accomplish a different task than the one it was originally trained on This paper proposed "adversarial reprogramming" of well-trained and fixed neural networks and show that adversarial reprogramming is less effective on untrained networks. The paper extends the idea of 'adversarial attacks' in supervised learning of NNs to a full repurposing of the solution of a trained net.