Anomaly detection discovers regular patterns in unlabeled data and identifies the non-conforming data points, which in some cases are the result of malicious attacks by adversaries. Learners such as One-Class Support Vector Machines (OCSVMs) have been successfully in anomaly detection, yet their performance may degrade significantly in the presence of sophisticated adversaries, who target the algorithm itself by compromising the integrity of the training data. With the rise in the use of machine learning in mission critical day-to-day activities where errors may have significant consequences, it is imperative that machine learning systems are made secure. To address this, we propose a defense mechanism that is based on a contraction of the data, and we test its effectiveness using OCSVMs. The proposed approach introduces a layer of uncertainty on top of the OCSVM learner, making it infeasible for the adversary to guess the specific configuration of the learner. We theoretically analyze the effects of adversarial perturbations on the separating margin of OCSVMs and provide empirical evidence on several benchmark datasets, which show that by carefully contracting the data in low dimensional spaces, we can successfully identify adversarial samples that would not have been identifiable in the original dimensional space. The numerical results show that the proposed method improves OCSVMs performance significantly (2-7%) Anomaly detection refers to the problem of discovering patterns in data and identifying data points that do not conform to the learned patterns. These non-conforming data points are often referred to as anomalies or outliers. Anomaly detection has numerous applications in a variety of domains such as network intrusion detection, credit card fraud detection, and spam filtering. It is an important problem since the presence of anomalies may indicate malicious attacks that could disrupt mission critical operations. Many machine learning methods, such as One-Class Support Vector Machines (OCSVM) BID14 , have been proven to be effective in anomaly detection applications. Although they are designed to withstand the effects of random noise in data, when adversaries deliberately alter the input data and compromise their integrity, the performance of these learning algorithms may degrade significantly.Anomaly detection systems are often deployed in environments where the data naturally evolves. In such situations, the models need to be retrained periodically, in contrast to many conventional machine learning applications, where the current and future data is assumed to have identical properties. This periodic training may allow adversaries to gradually inject malicious data to diminish the decision making capabilities of the learning algorithms BID8 . The aim of the adversaries may be to avoid the detection of attacks or to decrease the performance of the learning system BID8 . To achieve these aims, adversaries can undermine learning algorithms in several ways. For instance, they may manipulate the training data if it is gathered from the real operation of a system (e.g., spam filtering, firewall, anti-virus, etc.) and force the learning algorithm to learn a distorted representation that is favorable to them.A sophisticated adversary has the capacity to conduct an attack in numerous ways. Hence, it is not feasible to provide a general analysis that covers the whole range of attacks, across different machine learning algorithms. In this work, we explore the following key question: Is it possible to make OCSVMs more resistant against adversarial attacks which target the integrity of the training data through distortions? . If an adversary can maliciously perturb the input data used by a learning algorithm, they can force the learner to learn a model that is favorable to them. It has become imperative to secure machine learning systems against such adversaries due to the recent increase of automation in many day to day applications. In the context of image recognition, the perturbations caused by an adversary are usually imperceptible to humans, but they can force a learned model to mis-classify the perturbed images with high confidence. As BID5 have shown, with the emergence of self driving vehicles, an adversary could alter a "S-T-O-P" road sign in such a way that a vehicle (learning system) would reliably classify it as a "Speed Limit 45" sign. Such perturbations could be imperceptible to humans and could result in the loss of human lives.Our goal is to utilize a nonlinear data projection based algorithm to increase the attack resistance of OCSVMs against an adversarial opponent under realistic assumptions. The theory of nonlinear random projections facilitates large-scale, data-oriented, multi-agent decisions by reducing the number of optimization parameters and variables. Recent work in the literature shows that nonlinear random projections improve the training and evaluation times of kernel machines, without significantly compromising the accuracy of the trained models BID13 BID4 . In this paper, we show that under adversarial conditions, selective nonlinear random projections can be leveraged to increase the attack resistance of OCSVMs as well.A dataset X ∈ R n×d that is projected using a carefully chosen projection matrix A ∈ R d×r comprised of random elements that are normally distributed, would have its pairwise Euclidean distances preserved with high probability in the projected space XA BID9 . Therefore, the properties of the original data distribution would be present in the projected dataset with only minor perturbations. Note that here r is the dimension to which the data is nonlinearly projected and r < d. Since the elements of A are drawn randomly, the learner obtains an additional layer of security as it becomes virtually impossible for the adversary to guess the projection mechanism used by the learner due to the search space becoming unbounded.More formally, let w * pd 2 be the length of the weight vector of the OCSVM in the transformed space, after solving the corresponding optimization problem that includes the distortion made by the adversary and the nonlinear random projection. Let w * p 2 be the length of the weight vector in the transformed space, where there is no adversary present. Since the learner cannot distinguish between the original data and the distorted data, the learner would not have the ability to explicitly calculate w * p 2 . Therefore, for reasonable values of r and small distortions D, we prove in this paper that w * p 2 is bounded above: DISPLAYFORM0 The main contributions of this work are summarized as follows. We derive analytically an upper bound on the length of the weight vector of a OCSVM trained on an undistorted dataset that has been nonlinearly transformed to a lower dimensional space. In addition, the resistance added by nonlinear data transformations against an adversarial opponent is studied through numerical experiments on several benchmark datasets. We believe that our proposed approach can (i) increase the attack resistance of OCSVMs under adversarial conditions, and (ii) give the learner a significant advantage from a security perspective by adding a layer of unpredictability through the randomness of the data transformation in a selective direction. The experimental evaluation presented in the following section demonstrates the effectiveness of our proposed defense mechanism on three benchmark datasets: MNIST, CIFAR-10, and SVHN. We compare the performance of OCSVMs in conjunction with nonlinear random projections, when an active adversary is conducting a directed attack by maliciously distorting the data. We observe that the f-scores across the dimensions decrease between train C |test D and train D |test D . This indicates that a OCSVM trained on clean data can identify adversarial samples better than a OCSVM trained on distorted data. Consequently this shows that OCSVMs are not immune to integrity attacks by design, and by carefully crafting adversarial data points, adversaries can manipulate OCSVMs to learn models that are favorable to them.A comparison between f-scores of train D |test C and train D |test D shows that, as the dimension is reduced from the original dimension, the f-scores increase, but as we reduce the dimension further, the f-scores begin to decrease. The increase in f-score confirms that by projecting data to a lower dimensional space using a carefully selected direction, we can identify adversarial samples that would not have been identifiable in the original feature space. This is confirmed by the graphs in the second row, which show the false positive rate of the OCSVMs under integrity attacks (i.e., number of anomalies that are undetected). We find that there is a significant improvement in detecting adversarial samples under the proposed approach (e.g., 23% on CIFAR-10 and 31% on MNIST).When the dimensions are reduced below a certain dataset dependent threshold the OCSVM performance starts to decline (e.g., SVHN 1,500 vs 463). We postulate that the explanation of this effect is the reduction in distance between classes (in this case perturbed anomalies and normal data points) with the dimension. As we reduce the dimension of the transformation, we are able to reduce the effects of the adversarial datapoints. But at the same time, there is a significant loss of useful information due to the dimensionality reduction. Due to the interplay between these two factors, the performance of OCSVMs reduces as we decrease the dimension beyond a certain threshold. Finally, TAB2 shows the effectiveness of the bound derived in Theorem 1. The results show the consistency of the upper bound, which becomes tighter under dimension reduction.In summary, the above experiments demonstrate that, (i) OCSVMs are vulnerable to adversarial attacks on integrity, (ii) by projecting a distorted dataset to a lower dimension in an appropriate direction we can increase the robustness of the learned model w.r.t. integrity attacks, (iii) the performance, in terms of f-score, starts to decline when the dimensionality is reduced beyond a certain threshold, and (iv) the performance in the projected spaces, when there are no attacks on integrity, is comparable to that in the original dimensional space, but with less computational burden. This paper presents a theoretical and experimental investigation based on a unique combination of unsupervised anomaly detection, using OCSVMs and random projections for dimensionality reduction in the presence of a sophisticated adversary. Our numerical analysis focuses on two main aspects: the performance of OCSVMs in lower dimensional spaces under adversarial conditions and the impact of nonlinear random projections on the robustness of OCSVMs w.r.t. adversarial perturbations. The results suggest that OCSVMs can be significantly affected if an adversary has access to the data on which they are trained. For each dataset, with very high probability, there is at least one dimensionality and projection direction that results in a OCSVM that is able to identify adversarial samples that would not have been identifiable by a OCSVM in the original dimensional space. Due to the layer of uncertainty added by the randomness of the projection, our approach makes the learning system more secure by making it virtually impossible for an adversary to guess the underlying details of the learner. Therefore, our approach can be utilized to make a learning system secure by, (i) reducing the impact of possible adversarial perturbations by contracting, and moving the normal data cloud away from the origin in the projected space, and (ii) making the search space of the adversary unbounded by adding a layer of randomness.Since data contraction is at the core of our proposed approach, for our future work we would like to investigate whether our approach will still hold if used with other learning algorithms. One major question that arises from this work is how to optimally select the number of dimensions to transform the data to. We are currently exploring the possibility of using the intrinsic dimensionality of datasets to address this problem. Since there is a clear information asymmetry between the adversary and learner (due to the randomness), this problem provides a good foundation to explore game-theoretical formulations of anomaly detection and adversarial learning problems under dimensionality reduction techniques. We also plan to study "boiling frog" type of attacks, where the adversary gradually injects malicious data over time.A PROOFS Definition 1. Let X ∈ R n×d be the matrix that contains the training data. Similarly, define D ∈ R n×d as the matrix that contains the distortions made by the Adversary. Let A ∈ R d×r be the projection matrix where each element is an i.i.d. N (0, 1) random variable. Define b as a 1 × r row vector where each element is drawn uniformly from [0, 2π] . Using these variables, we define C ∈ R n×r , where the element at row i column j takes the following form.C i,j = cos DISPLAYFORM0 C i,j = cos DISPLAYFORM1 Similarly, we define the matrices C X , C D , S X , S D as follows, Proof: (of Theorem 1) Letα be the vector achieving the optimal solution in the projected space when adversarial distortions are present. Then, the solution for the primal problem in the projected space with adversarial distortions, defined as wSince the optimization problem is a minimization problem, as shown in (2), the optimal solution for the OCSVM without any distortion (i.e., α * ) would give a value less than or equal to the value given byα. Thus, DISPLAYFORM2 Define w * p as the primal solution optimization in the projected space, if there were no adversarial perturbations present, therefore DISPLAYFORM3 B RESULTS
A novel method to increase the resistance of OCSVMs against targeted, integrity attacks by selective nonlinear transformations of data to lower dimensions. The authors propose a defense against attacks on the security of one-class SVM based anomaly detectors This paper explores how random projections can be used to make OCSVM robust to adversarially perturbed training data.