The log marginal likelihood for Gaussian Process regression is calculated according to Chapter 5 of the Rasmussen and Williams GPML book:

$log\ p(y|X,\theta) = -\frac{1}{2}y^TK_y^{-1}y-\frac{1}{2}log|K_y|-\frac{n}{2}log2\pi$

It is straightforward to get a single log marginal likelihood value when the regression output is one dimension. But when it comes to multiple-output regression, the first term on the right side of the above equation is actually a matrix instead of a single value:


So I am wondering how to handle this situation. I notice in many Gaussian Process implementations, people get the log marginal likelihood by summing the posterior pdf of each training sample, which is similar to the "LOO-CV" concept mentioned in Chapter 5 of the GPML book, but without really leaving one out during training. Is this a reasonable way to calculate the log marginal likelihood in Gaussian Process context? Thanks for any feedback in advance!


Terms like $y^tMy$ are scalar. It's known as a quadratic form. In your case, $M=K_y^{-1}$.

I assume that $K$ is a function of $\theta$, the vector of unknown parameters. You grab the partial derivatives with respect to the $\theta$ vector and solve for zero. Alternatively, one could maximize the log likelihood directly by using a numeric optimization algorithm.

  • $\begingroup$ Thanks for reply! However, I think in the multiple-output regression situation, $y$ is actually a matrix with the dimension N X OutputDim, where N is the number of the training samples, so the term $-\frac{1}{2}y^TK_y^{-1}y$ is finally a OutputDim X OutputDim matrix instead of a scalar here, or do I misunderstand something? $\endgroup$ – mikegao88 Nov 23 '15 at 6:42
  • 1
    $\begingroup$ I think that y is a vector of scalar responses. chapter 5.4 looks at the same model as chapter 2, which involves a scalar response. the feature space is higher dimensional. $\endgroup$ – Placidia Nov 23 '15 at 17:44

Your Answer

