Awareness of uncertainty in classification using a multivariate model
  and multi-views
Alexey Kornaev Elena Kornaeva Oleg Ivanov Ilya Pershin Danis Alukaev
One of the ways to make artificial intelligence more natural is to give it
some room for doubt. Two main questions should be resolved in that way. First,
how to train a model to estimate uncertainties of its own predictions? And
then, what to do with the uncertain predictions if they appear? First, we
proposed an uncertainty-aware negative log-likelihood loss for the case of
N-dimensional multivariate normal distribution with spherical variance matrix
to the solution of N-classes classification tasks. The loss is similar to the
heteroscedastic regression loss. The proposed model regularizes uncertain
predictions, and trains to calculate both the predictions and their uncertainty
estimations. The model fits well with the label smoothing technique. Second, we
expanded the limits of data augmentation at the training and test stages, and
made the trained model to give multiple predictions for a given number of
augmented versions of each test sample. Given the multi-view predictions
together with their uncertainties and confidences, we proposed several methods
to calculate final predictions, including mode values and bin counts with soft
and hard weights. For the latter method, we formalized the model tuning task in
the form of multimodal optimization with non-differentiable criteria of maximum
accuracy, and applied particle swarm optimization to solve the tuning task. The
proposed methodology was tested using CIFAR-10 dataset with clean and noisy
labels and demonstrated good results in comparison with other uncertainty
estimation methods related to sample selection, co-teaching, and label
smoothing.