The statistics of the real visual world presents a long-tailed distribution: a few classes have significantly more training instances than the remaining classes in a dataset. This is because the real visual world has a few classes that are common while others are rare. Unfortunately, the performance of a convolutional neural network is typically unsatisfactory when trained using a long-tailed dataset. To alleviate this issue, we propose a method that discriminatively learns an embedding in which a simple Bayesian classifier can balance the class-priors to generalize well for rare classes. To this end, the proposed approach uses a Gaussian mixture model to factor out class-likelihoods and class-priors in a long-tailed dataset. The proposed method is simple and easy-to-implement in existing deep learning frameworks. Experiments on publicly available datasets show that the proposed approach improves the performance on classes with few training instances, while maintaining a comparable performance to the state-of-the-art on classes with abundant training examples. Deep convolutional neural networks (CNN) have achieved impressive results in large-scale visual recognition tasks BID13 BID25 BID29 BID18 BID8 BID10 . However, despite the significant impact in visual perception, the vast majority of these advancements learn from artificially balanced largescale datasets that are not representative of the real visual world BID19 BID5 BID3 BID21 BID15 BID23 . The statistics of the real visual world follow a long-tailed distribution BID36 BID32 BID24 BID33 BID34 . This means that a few classes are predominant in the world while others are rare. Consequently, representative real-world datasets have a few classes with significantly more training instances than the remaining classes in the set; see Fig. 1 (a) for an illustration of a long-tailed dataset. We refer to classes with abundant training instances as classes in the head, and unrepresented classes as classes in the tail.As BID32 note, the main motivation for visual recognition is to understand and learn from the real visual world. Thus, while the state-of-the-art can challenge humans in visual recognition tasks, it misses a mechanism that effectively learns from long-tailed datasets. As BID32 found, training models using long-tailed datasets often leads to unsatisfying performance. This is because classifiers tend to generalize well for classes in the head, but lack generalization for classes in the tail.To alleviate this issue, learned classifiers need to generalize for classes in the tail while maintaining a good performance for all the classes. Recent efforts that aim to learn from long-tailed datasets consider penalities in the optimization-learning problem BID9 , sampling-based methods BID7 , and transfer-learning algorithms BID33 BID34 . In contrast with these solutions, the proposed method aims to learn an embedding in which the distribution of the real visual world allows a simple Bayesian classifier to predict robustly given a long-tailed dataset.Long-tailed datasets have class-prior statistics that heavily skew towards classes in the head. This skew can bias classifiers towards classes in the head, and consequently can reduce generalization for classes in the tail. To remove this skew, we appeal to Bayesian classifiers that can explicitly Figure 1: (a) The real visual world yields long-tailed datasets. Classes in the head are common (e.g., cats) while classes in the tail are rare (e.g., white reindeers). (b) The proposed approach builds a generative (Bayesian) classifier over a learned embedding to compute class-posterior probabilities. In an empirical Bayesian framework, posteriors are computed through class likelihoods and priors fit to the data (e.g., sample means, variances, and counts assuming Gaussian Mixture Models). We introduce an end-to-end pipeline for jointly learning embeddings and Bayesian models built upon them. (c) Bayesian models are particularly well-suited for long-tailed datasets because class priors and likelihoods can be fixed to be uniform and isotropic, ensuring that the learned representation is balanced across the head and tail.factor out the likelihood and prior when computing posteriors over class labels. Thus, the main goal of this work is to learn a feature embedding in which class prior statistics do not affect/skew class likelihoods. The proposed approach uses a simple Gaussian mixture model (GMM) to describe the statistics of a long-tailed dataset. This is because it enables a clean factorization of the class-likelihoods and class-priors. Moreover, it easily fits within an empirical Bayesian classification framework, because a GMM enables the computation of closed-form maximum likelihood estimation (MLE) of class-specific means, covariance matrices, and priors. We show that such closed-form estimates can be integrated into existing deep learning optimizers without much effort. By fixing the covariance matrices of all the classes to be the identity and the priors over each class to be uniform, we can explicitly enforce that both rare classes in the tail and dominant classes in the head have equal weight for Bayesian classification. In simple terms: we learn a discriminative embedding of training data such that Bayesian classifiers with balanced priors produce accurate class posteriors. As a point of clarity, the proposed approach does not learn an embedding in the traditional Bayesian sense, which might define a prior distribution over embeddings that is then combined with training data to produce a posterior embedding. Rather, it learns a single embedding that is discriminatively trained to produce accurate features for Bayesian classifiers. See Fig. 1 for an illustration about the proposed approach.A GMM not only is useful for learning an embedding using a long-tailed dataset, but also provides flexibility at the evaluation stage. This is because it enables the measurement of generalization for classes in the tail by simply setting equal class-prior probabilities. In addition, it enables the possibility of giving more importance to the most frequent classes by adjusting their respective class-prior probabilities.In sum, the proposed approach aims to learn an embedding in which a GMM enables a Bayesian classifier to generalize well for classes in the tail by balancing out class-priors. The proposed method is simple, easy-to-train using deep learning frameworks, and increases classification performance for classes in the tail. The experiments on publicly available datasets show that this approach tends to perform better on classes in the tail than the competing methods, while performing comparable to the state-of-the-art on classes with abundant training instances. Other probabilistic models: Our analysis and experiments focus on Gaussian Mixture Models, but the general learning problem from Eq. (4) holds for other probabilistic models. For example, deep embeddings can be learned for rectified (nonnegative) or binary features BID0 BID4 . For such embeddings, likelihood models based on rectified Gaussians or multivariate Bernoulli distributions may be more appropriate BID27 BID30 . Such models do not appear to have closed form maximum likelihood estimates, and so may be challenging to formulate precisely as a constrained optimization problem.Relationship to softmax: The GMM-based formulation has a direct relationship with softmax classifiers. This relationship can be obtained by expanding the squared distance terms in the classposterior probability, yielding the following: DISPLAYFORM0 where v j = µ j and DISPLAYFORM1 2 is a common term between the numerator and denominator. This relationship thus indicates that the proposed approach fits linear classifiers with restricted biases. This relationship is useful for an easy implementation in many deep learning frameworks. This is because this approach can be implemented using a dense layer without the bias terms. In addition, this relationship shows that the proposed approach requires fewer parametersto-learn in comparison with classical CNN-softmax models. An more intuitive comparison between GMMs and softmax classifiers can be made with respect to to their parameter updates. Intuitively, during softmax training, an "easy" example of a class will not generate a model update. In some sense, this might be considered paradoxical. When children learn a new concept (for say, a neverbefore-seen animal), they tend to be presented with an easy or "protypical" example. On the other hand, an easy example of a class will change its centroid, generating a signal for learning -see FIG1 . This work introduced a method that improves the classification performance for classes in the tail. The proposed approach is based on a Gaussian mixture model that allows a Bayesian classifier to represent the distribution of a long-tailed dataset and to compute the class-prediction probabilities. The experiments on publicly available dataset show that the proposed approach tends to increase the classification accuracy for classes in the tail while maintaining a comparable accuracy to that of a softmax classifier for classes in the head. In addition, this work introduced an evaluation method for methods that tackle the learning of concepts from a long-tailed dataset. Finally, this work demon-strated that class-centroid approaches overall tend to generalize well for classes in the tail while maintaining a comparable performance to that of a softmax classifiers for classes in the head.
Approach to improve classification accuracy on classes in the tail. The main goal of this paper is to learn a ConvNet classifier which performs better for classes in the tail of the class occurrence distribution. Proposal for a Bayesian framework with a Gaussian mixture model to address an issue in classification applications, that the number of training data from different classes is unbalanced.