This report introduces a training and recognition scheme, in which classification is realized via class-wise discerning. Trained with datasets whose labels are randomly shuffled except for one class of interest, a neural network learns class-wise parameter values, and remolds itself from a feature sorter into feature filters, each of which discerns objects belonging to one of the classes only. Classification of an input can be inferred from the maximum response of the filters. A multiple check with multiple versions of filters can diminish fluctuation and yields better performance. This scheme of discerning, maximum response and multiple check is a method of general viability to improve performance of feedforward networks, and the filter training itself is a promising feature abstraction procedure. In contrast to the direct sorting, the scheme mimics the classification process mediated by a series of one component picking. Suppose given a task of sorting a bunch of colored balls, we can usually do it in two ways. One is to randomly pick up a ball and deposit it to one of the groups according to its color. Or we can collect balls of the same color at a time until the last two colors are separated. Standard neural networks for classification tasks work in the former manner, for which it is an implied principle that every class should be on an equal footing and biases are harmful. This is the reason why the training data contain subsets of identical sample number or close in size for each class. The scheme introduced in this work is an analogue of the latter, in which the networks do not respond equally to features of different classes. Rather, they function more like filters, and ascription of an object is inferred from how strong their responses are.The filter networks are obtained by filter training, from which we have a group of networks or more specifically a batch of parameter values, number of which is equal to the number of classes. Ensemblization appears to be a built-in trait of this recognition scheme. Since a prediction should not refer to a preassigned label, a quantitative evaluation of how each filter responds to an input should be defined on an equal footing. As long as one of the alternative filters scores higher than the correct one, the prediction is wrong. Fluctuation makes the correct filter the weaker in this one to many contest. To do it a favour we introduce another hierarchy of ensemblization that is a batch of versions for the filters, in the hope that the correct filter could be the overall winner in this tournament. These three steps constitute a classification procedure of discerning, maximum response, and multiple check (DMM).The DMM scheme can improve accuracy of mediocre networks, networks already having high accuracy, and those trained with small-scale data. Fundamental reason for the increase is that in the filter training a multiclass problem is reduced to a pseudo binary classification, the class of interest and the others. Intuitively , we have the feeling that telling a component from a mixture is easier than sorting all the ingredients. The pseudo reduction mitigates workload. Depending on capacity of networks and amount of training samples, the increase of accuracy varies. Nevertheless , as performance improvement due to the mitigation is almost sure, the scheme can be a general route to enhance feedforward networks. Moreover, since a filter is specifically trained for one class, the filter training is a feature abstraction procedure in itself.In the following, we first investigate the mechanism of filter training and how it works through a toy model. From this classification of points in a 2D plane, we can clearly view how the decision boundaries are reshaped. Then, we give a probabilistic argument why the maximum response is a proper criterion to infer classification. After remarking on its relation with related works, we experiment MMD on the CIFAR-10 and MNIST datasets. Motivated by the classification process via one component discerning, we propose the filter training and maximum response. The multiple check can build up an increment of performance if the fluctuation in the responses are properly distributed among filter versions. How to deal with the two types of fluctuations is the major concern for it to work well. DMM constitutes a special ensemble learning scheme, which itself can be incorporated into other schemes as an additional hierarchy of ensemblization. It is beneficial if similar mechanism can be integrated into other network architectures, since task simplification is a common strategy in intellegence activities.
The proposed scheme mimics the classification process mediated by a series of one component picking. A method to increase accuracy of deep-nets on multi-class classification tasks seemingly by a reduction of multi-class to binary classification. A novel classification procedure of discerning, maxmum response, and multiple check to improve accuracy of mediocre networks and enhance feedforward networks.