Memory-Based Dual Gaussian Processes for Sequential Learning
Paul E. Chang Prakhar Verma S. T. John Arno Solin Mohammad Emtiyaz Khan
Sequential learning with Gaussian processes (GPs) is challenging when access
to past data is limited, for example, in continual and active learning. In such
cases, errors can accumulate over time due to inaccuracies in the posterior,
hyperparameters, and inducing points, making accurate learning challenging.
Here, we present a method to keep all such errors in check using the recently
proposed dual sparse variational GP. Our method enables accurate inference for
generic likelihoods and improves learning by actively building and updating a
memory of past data. We demonstrate its effectiveness in several applications
involving Bayesian optimization, active learning, and continual learning.