Convolutional neural networks (CNNs) have been successfully applied to many recognition and learning tasks using a universal recipe;  training a deep model on a very large dataset of supervised examples. However, this approach is rather restrictive in practice since collecting a large set of labeled images is very expensive. One way to ease this problem is coming up with smart ways for choosing images to be labelled from a  very large collection (i.e. active learning).

 Our empirical study suggests that many of the active learning heuristics in the literature are not effective when applied to CNNs when applied in batch setting. Inspired by these limitations, we define the problem of active learning as core-set selection, i.e. choosing set of points such that a model learned over the selected subset is competitive for the remaining data points. We further present a theoretical result characterizing the performance of any selected subset using the geometry of the datapoints. As an active learning algorithm, we choose the subset which is expected to yield best result according to our characterization. Our experiments show that the proposed method significantly outperforms existing approaches in image classification experiments by a large margin.
 Deep convolutional neural networks (CNNs) have shown unprecedented success in many areas of research in computer vision and pattern recognition, such as image classification, object detection, and scene segmentation. Although CNNs are universally successful in many tasks, they have a major drawback; they need a very large amount of labeled data to be able to learn their large number of parameters. More importantly, it is almost always better to have more data since the accuracy of CNNs is often not saturated with increasing dataset size. Hence, there is a constant desire to collect more and more data. Although this a desired behavior from an algorithmic perspective (higher representative power is typically better), labeling a dataset is a time consuming and an expensive task. These practical considerations raise a critical question: "what is the optimal way to choose data points to label such that the highest accuracy can be obtained given a fixed labeling budget." Active learning is one of the common paradigms to address this question.The goal of active learning is to find effective ways to choose data points to label, from a pool of unlabeled data points, in order to maximize the accuracy. Although it is not possible to obtain a universally good active learning strategy BID4 , there exist many heuristics BID38 which have been proven to be effective in practice. Active learning is typically an iterative process in which a model is learned at each iteration and a set of points is chosen to be labelled from a pool of unlabelled points using these aforementioned heuristics. We experiment with many of these heuristics in this paper and find them not effective when applied to CNNs. We argue that the main factor behind this ineffectiveness is the correlation caused via batch acquisition/sampling. In the classical setting, the active learning algorithms typically choose a single point at each iteration; however, this is not feasible for CNNs since i) a single point is likely to have no statistically significant impact on the accuracy due to the local optimization methods, and ii) each iteration requires a full training until convergence which makes it intractable to query labels one-by-one. Hence, it is necessary to query labels for a large subset at each iteration and it results in correlated samples even for moderately small subset sizes.In order to tailor an active learning method for the batch sampling case, we decided to define the active learning as core-set selection problem. Core-set selection problem aims to find a small subset given a large labeled dataset such that a model learned over the small subset is competitive over the whole dataset. Since we have no labels available, we perform the core-set selection without using the labels. In order to attack the unlabeled core-set problem for CNNs, we provide a rigorous bound between an average loss over any given subset of the dataset and the remaining data points via the geometry of the data points. As an active learning algorithm, we try to choose a subset such that this bound is minimized. Moreover, minimization of this bound turns out to be equivalent to the k-Center problem (Wolf, 2011) and we adopt an efficient approximate solution to this combinatorial optimization problem. We further study the behavior of our proposed algorithm empirically for the problem of image classification using three different datasets. Our empirical analysis demonstrates state-of-the-art performance by a large margin. We study the active learning problem for CNNs. Our empirical analysis showed that classical uncertainty based methods have limited applicability to the CNNs due to the correlations caused by batch sampling. We re-formulate the active learning problem as core-set selection and study the core-set problem for CNNs. We further validated our algorithm using an extensive empirical study. Empirical results on three datasets showed state-of-the-art performance by a large margin.A PROOF FOR LEMMA 1Proof. We will start with showing that softmax function defined over C class is , i = 1, 2, ..., C For brevity, we will denote f i (x) as f i . The Jacobian matrix will be, DISPLAYFORM0 Now, Frobenius norm of above matrix will be, DISPLAYFORM1 It is straightforward to show that f i = 1 C is the optimal solution for J * F = max . If we assume, i |w i,j | ≤ α ∀i, j, d, for any convolutional or fully connected layer, we can state: DISPLAYFORM2 On the other hand, using |a − b| ≤ | max(0, a) − max(0, a)| and the fact that max pool layer can be written as a convolutional layer such that only one weight is 1 and others are 0, we can state for ReLU and max-pool layers, DISPLAYFORM3 Combining with the Lipschitz constant of soft-max layer, B PROOF FOR THEOREM 1 DISPLAYFORM4 Before starting our proof, we state the Claim 1 from BID1 . Fix some p, p ∈ [0, 1] and y ∈ {0, 1}. Then, p y∼p (y = y ) ≤ p y∼p (y = y ) + |p − p |
We approach to the problem of active learning as a core-set selection problem and show that this approach is especially useful in the batch active learning setting which is crucial when training CNNs. The authors provide an algorithm-agnostic active learning algorithm for multi-class classification The paper proposes a batch mode active learning algorithm for CNN as a core-set problem which outperforms random sampling and uncertainty sampling. Studies active learning for convolutional neural networks and formulates the active learning problem as core-set selection and presents a novel strategy