On the Identifiability and Interpretability of Gaussian Process Models
Jiawen Chen Wancen Mu Yun Li Didong Li
In this paper, we critically examine the prevalent practice of using additive
mixtures of Mat\'ern kernels in single-output Gaussian process (GP) models and
explore the properties of multiplicative mixtures of Mat\'ern kernels for
multi-output GP models. For the single-output case, we derive a series of
theoretical results showing that the smoothness of a mixture of Mat\'ern
kernels is determined by the least smooth component and that a GP with such a
kernel is effectively equivalent to the least smooth kernel component.
Furthermore, we demonstrate that none of the mixing weights or parameters
within individual kernel components are identifiable. We then turn our
attention to multi-output GP models and analyze the identifiability of the
covariance matrix $A$ in the multiplicative kernel $K(x,y) = AK_0(x,y)$, where
$K_0$ is a standard single output kernel such as Mat\'ern. We show that $A$ is
identifiable up to a multiplicative constant, suggesting that multiplicative
mixtures are well suited for multi-output tasks. Our findings are supported by
extensive simulations and real applications for both single- and multi-output
settings. This work provides insight into kernel selection and interpretation
for GP models, emphasizing the importance of choosing appropriate kernel
structures for different tasks.