Meta-learning to Calibrate Gaussian Processes with Deep Kernels for
  Regression Uncertainty Estimation
Tomoharu Iwata Atsutoshi Kumagai
Although Gaussian processes (GPs) with deep kernels have been successfully
used for meta-learning in regression tasks, its uncertainty estimation
performance can be poor. We propose a meta-learning method for calibrating deep
kernel GPs for improving regression uncertainty estimation performance with a
limited number of training data. The proposed method meta-learns how to
calibrate uncertainty using data from various tasks by minimizing the test
expected calibration error, and uses the knowledge for unseen tasks. We design
our model such that the adaptation and calibration for each task can be
performed without iterative procedures, which enables effective meta-learning.
In particular, a task-specific uncalibrated output distribution is modeled by a
GP with a task-shared encoder network, and it is transformed to a calibrated
one using a cumulative density function of a task-specific Gaussian mixture
model (GMM). By integrating the GP and GMM into our neural network-based model,
we can meta-learn model parameters in an end-to-end fashion. Our experiments
demonstrate that the proposed method improves uncertainty estimation
performance while keeping high regression performance compared with the
existing methods using real-world datasets in few-shot settings.