
Few-shot learning trains image classifiers over datasets with few examples per category. 
 It poses challenges for the optimization algorithms, which typically require many examples to fine-tune the model parameters for new categories. 
 Distance-learning-based approaches avoid the optimization issue by embedding the images into a metric space and applying the nearest neighbor classifier for new categories. In this paper, we propose to exploit the object-level relation to learn the image relation feature, which is converted into a distance directly.
 For a new category, even though its images are not seen by the model, some objects may appear in the training images. Hence, object-level relation is useful for inferring the relation of images from unseen categories. Consequently, our model generalizes well for new categories without fine-tuning.
 Experimental results on benchmark datasets show that our approach outperforms state-of-the-art methods. Real-world data typically follows power-law distributions, where the majority of the data categories have only a small number of examples. For instance, to train an image classifier for food images, one would probably crawl few images for some local dishes. Similarly, there are few images for new products, e.g. new toys. However, state-of-the-art image classifiers, i.e. deep convolutional neural networks (ConvNets) BID7 , are hungry for data. The benchmark datasets for ConvNets, including CIFAR10 and ImageNet BID0 , usually have more than 1000 images per category. Fine-tuning ConvNets BID18 by transferring the knowledge (i.e. parameters) learned from a big dataset could alleviate the gap, but still fails to resolve the issue. This is because the widely used gradient-based optimization algorithms need many iterations over plenty of examples to adapt the ConvNets (with a large number of parameters) for new categories.Two types of approaches have been proposed towards addressing the above issue. They are referred as few-shot image classification, which trains classifiers over datasets with few (e.g. less than 20) examples per category. The first set of approaches BID12 ; ; BID2 are based on meta-learning. They train a meta learner to guide the optimization of the classifier for the new categories. They improve the optimization by providing a good initialization BID2 , an adaptive learning rate or even replacing the gradient-based optimization method BID12 . The second set of approaches ; BID17 ; BID13 ; BID15 ; BID16 are based on embedding learning. They learn an embedding function to project the images into a space and then classify images from new categories through the nearest neighbour search. No fine-tuning is required as the nearest neighbour classifier is non-parametric. The embedding functions are vital to the classification accuracy, which must be general enough to extract good embedding features for evaluating the distance/similarity between images belonging to the unseen categories.In this paper, we propose a new few-short learning approach. It is motivated by the observation that human beings are pretty good at few-shot learning. Take the Segway in FIG0 as an example BID6 ) although the Segway could be new to us, we are familiar with its components, e.g. wheels, which are similar to those of the motors or electric scooters. Hence, we know Segway is a traffic tool for riding. Moreover, we are able to analyze the image by decomposing it. For example, we are aware of the relationship between Segway and rider. This kind of relationship-awareness helps in recognition when we see a different rider with another Segway. However, existing methods take each image as a whole without exploiting the object-level information including relation.Based on the above observation, we design our learning model with two parts, namely the relation extraction network and the distance learning network. We draw inspiration from the relation network BID14 . In particular, we compare the objects from two images instead of a single image as BID14 in order to extract the relation between images. The relation is converted into a similarity score. We expect the object relationship to play a crucial role in determining the image relation for distinguishing images from different categories. The training is conducted in episodes, each of which is constructed in the same way as in the test, i.e. with few examples for each category. After training, we extract the relation feature vectors among the query image (to be classified) and each labelled image from the test dataset. Nearest neighbour classifier is then applied with the similarity score calculated from the relation feature vector. Extensive experiments on benchmark datasets confirm the superiority of our approach in terms of classification accuracy against existing work. In this paper, we exploit the object-level relation to infer the image relation. In particular, we consider each 'pixel' on the feature map as an object in the input image, and use the values across all channels as the object feature. In fact, one 'pixel' corresponds to one patch of the original image. Small patches may not contain any objects, while in big patches, there could be multiple objects. In the extreme case where the feature map size is 1x1, the corresponding patch is the whole image. Our model is then equivalent to LearningToCompare Sung et al. (2017) . In fact, we capture object pairs from different locations, whereas LearningToCompare is restricted to element-wise match at the same spatial location. The effectiveness of our approach from the experimental study confirms that the relation extracted from multiple local patches is useful for determining the image relation. Particularly, we do one more set of comparison against Learning2Compare since it has similar architecture as our model. We change the input image size of Learning2Compare to 224x224, i.e. , the same as our model. The accuracy for 5-way 1-shot and 5-way 5-shot is 50.16% and 65.98% respectively. In addition, we compare effect of the feature map size of our model. We vary the feature map size as 1x1, 3x3, 5x5, 7x7 and 9x9. The corresponding accuracy of 5-way 5-shot classification is 64.55%, 67.78%, 69.68%, 69.82%, 70.90%. The improvement becomes marginal for bigger sizes. We can see that with the increasing of objects number (feature map size), the performance improves. It indicates that the objects-level relation does make a difference.The basic idea behind our model is to aggregate the local information for global reasoning. In this paper, we consider the spatial local information. However, the framework is extensible for other local information. For example, if we treat different feature maps as different aspects (or features) of the image, in Figure 2 , we can compare (combine) these feature maps from two images to get c × c local relation features, each of size w × h + w × h. Similarly, we can compare the features of different attributes of each class with the feature maps to do zero-shot learning. It compares the local text information with local image information. ConvNets have shown great success for image classification with many examples per category. However, few-shot learning is challenging because the training algorithms of ConvNets, i.e. gradient based optimization algorithms, require many iterations to fine-tune the parameters over a lot of examples for new image classes. In this paper, we avoid the fine-tuning step by training a model that is general to learn the relation of images from unseen categories. We observe that the object-level relation persists across training and test images, although the relation of images from test datasets is unseen. Therefore, we propose to exploit object-level relation to infer the image relation. In particular, object features are compared (combined) and transformed to extract the image relation feature, which is applied directly for similarity learning. Using the learned similarity from the relation feature, our approach outperforms existing algorithms for few-shot image classification tasks.
Few-shot learning by exploiting the object-level relation to learn the image-level relation (similarity) This paper deals with the problem of few-shot learning by proposing an embedding-based approach that learns to compare object-level features between support and query set examples Proposes a few shot learning method that exploits the object-level relation between different images based on neared neighbor search and concatenates feature maps of two input images into one feature map