Take the 2-minute tour ×

I'm taking a machine learning course and it involves a lot of matrix computation like compute the derivatives of a matrix with respect to a vector term. In my linear algebra course these material is not covered and I browsed some book in the school library but didn't find something relevant to my problem. So can you recommend me a book which covers these matrix computations?


share|improve this question
Related: math.stackexchange.com/questions/135008/… –  Belgi Nov 2 '12 at 0:27

4 Answers 4

up vote 2 down vote accepted

I find the Matrix Cookbook handy. It's not a textbook, rather, to quote the introduction:


share|improve this answer
This seems good because it has a whole chapter on derivatives. "This section is covering diﬀerentiation of a number of expressions with respect to a matrix X." –  littleO Nov 6 '12 at 10:45

I recommend two books

  1. Matrix Algebra from a statistician's perspective by David Harville. (chapter 15 covers derivatives of matrices with respect to matrices.)
  2. Matrix Differential Calculus with Applications in Statistics and Econometrics by Jan Magnus and Heinz Neudecker. (a huge portion of the book is about that particular question of differentiation.)

These two books are also excellent because they come with two additional monographs with a lot of solved problems (check for instance Matrix Algebra (Econometric Exercises, Vol. 1) by Karim Abadir and Jan Magnus which has a lot of solved problems on matrix differentiation). Also they have the additional advantage that they are directly relevant to machine learning.

share|improve this answer

I suggest Matrix computation by Golub and van Loan.

share|improve this answer
This needs about +10 –  Arkamis Nov 2 '12 at 0:37

Bishop's excellent machine learning textbook, Pattern Recognition and Machine Learning, has an appendix on "Properties of Matrices". One section in this appendix (p. 697) is about Matrix Derivatives, and discusses formulas like this:

\begin{equation} \frac{\partial}{\partial A} \text{Tr}(AB) = B^T \end{equation}


\begin{equation} \frac{\partial}{\partial A} \ln | A | = \left(A^{-1} \right)^T, \end{equation} for example.

Peter Lax, one of the greatest living mathematicians, has an excellent book called Linear Algebra and its Applications. Chapter 9 is entitled "Calculus of Vector- and Matrix-Valued Functions".

share|improve this answer

Your Answer


