Accuracy-Preserving Calibration via Statistical Modeling on Probability
  Simplex
Yasushi Esaki Akihiro Nakamura Keisuke Kawano Ryoko Tokuhisa Takuro Kutsuna
Classification models based on deep neural networks (DNNs) must be calibrated
to measure the reliability of predictions. Some recent calibration methods have
employed a probabilistic model on the probability simplex. However, these
calibration methods cannot preserve the accuracy of pre-trained models, even
those with a high classification accuracy. We propose an accuracy-preserving
calibration method using the Concrete distribution as the probabilistic model
on the probability simplex. We theoretically prove that a DNN model trained on
cross-entropy loss has optimality as the parameter of the Concrete
distribution. We also propose an efficient method that synthetically generates
samples for training probabilistic models on the probability simplex. We
demonstrate that the proposed method can outperform previous methods in
accuracy-preserving calibration tasks using benchmarks.