I'm reading through these notes that claim that if we expand the log of the multivariate gaussian distribution we get the following:

$$log(N(x_n|\mu_k, \Sigma_k)) = -\frac{1}{2} (x_n - \mu_k)^T \Sigma_k^{-1}(x_n - \mu_k) -\frac{1}{2} log|\Sigma_k| -\frac{D}{2} log(2\pi)$$

This part makes sense, however upon expansion it says we have this:

$$log(N(x_n|\mu_k, \Sigma_k)) = \mu_k^{T} \Sigma_k^{-1}x_n - \frac{1}{2} \mu_k^T \Sigma_k^{-1}\mu_k$$

I'm not sure how expanding the top function results in the bottom one, I'm fairly certain $-\frac{D}{2} log(2\pi)$ was removed because eventually we want to maximize over k but i'm not sure about the rest of it...

any help would be appreciated.

Edit: The formula for a multivariate gaussian is:

$$N(x_n|\mu_k, \Sigma_k) = \frac{1}{(2\pi)^{\frac{D}{2}} |\Sigma|^{\frac{1}{2}}} e^{- \frac{1}{2} (x_n - \mu_k)^T \Sigma^{-1}(x_n - \mu_k)}$$


Ignoring all the subscripts, and writing $A:=\Sigma^{-1}$, expand as follows: $$(x-\mu)^TA(x-\mu)=x^TAx-x^TA\mu-\mu^TAx+\mu^TA\mu$$ All the terms on the RHS are scalars since they have dimension $(1\times D)\times (D\times D)\times (D\times 1)$. Take the transpose of the second term and we see it's equal to the third term (since the covariance matrix $A$ is symmetric). This now simplifies to your form if we're throwing away (i.e., treating as constant) any term that doesn't involve $\mu$. (This would be appropriate if you were trying to maximize the log likelihood over $\mu$.)

  • $\begingroup$ That makes sense, but why is $-x^T A \mu$ missing from the formula then? because $\mu$ is included in it but it's not in the formula that i have $\endgroup$ – Ghazal Feb 28 '16 at 6:55
  • $\begingroup$ As mentioned above, $-x^TA\mu$ is the second term in the RHS. It equals its transpose, which is $-\mu^TAx$. Now add it to the third term in the RHS, to get $-2\mu^TAx$. Finally multiply through by $-1/2$ (which I've left out all along) to get $\mu^TAx-\frac12\mu^TA\mu$. $\endgroup$ – grand_chat Feb 28 '16 at 8:24
  • $\begingroup$ I see, thank you so much $\endgroup$ – Ghazal Feb 28 '16 at 16:17

Your Answer

