How to calculate a projection matrix for nonnegative constrained least squares?
<p>Suppose we have a data vector <strong>z</strong> in R^p and a training data matrix <strong>X</strong> in R^(p*N), where N (N>p) is the number of samples in the training data matrix. </p>

<p>If we'd like to find the projection of <strong>z</strong> to the linear subspace spanned by the columns of <strong>X</strong>, then we can solve the following unconstrained problem</p>

<p>min  ||<strong>z</strong> - <strong>X</strong> <strong>b</strong>||^2.</p>

<p>The least squared estimation of <strong>b</strong> is (<strong>X</strong> <strong>X</strong>')^-1 <strong>X</strong> <strong>z</strong>. Thus the projection of <strong>z</strong> to the subspace can be written as <strong>X</strong>(<strong>X</strong> <strong>X</strong>')^-1 <strong>X</strong> <strong>z</strong> = <strong>P</strong> <strong>z</strong>. <strong>P</strong> = <strong>X</strong>(<strong>X</strong> <strong>X</strong>')^-1 <strong>X</strong> is a projection matrix satisfying <strong>P</strong>' = <strong>P</strong> and <strong>P</strong>^2 = <strong>P</strong>.</p>

<p>If we add nonnegative constraint to <strong>b</strong>, then the above optimisation problem becomes</p>

<p>min  ||<strong>z</strong> - <strong>X</strong> <strong>b</strong>||^2, s.t. every element in <strong>b</strong> is nonnegative.</p>

<p>The Lagrangian is ||<strong>z</strong> - <strong>X</strong> <strong>b</strong>||^2 - <strong>v</strong>^T <strong>b</strong>, where <strong>v</strong> is an N-dimensional vector. Setting the first order derivative of the Lagrangian w.r.t <strong>b</strong> to zero, we find the eistimation of <strong>b</strong>: (<strong>X</strong> <strong>X</strong>')^-1 (<strong>X</strong>' <strong>z</strong> + 1/2 <strong>v</strong>). Thus the projection of <strong>z</strong> to the subspace is <strong>X</strong> (<strong>X</strong> <strong>X</strong>')^-1 (<strong>X</strong>' <strong>z</strong> + 1/2 <strong>v</strong>).</p>

<p>My question now can be described as follows. </p>

<p>Is there any method that can approximate the projection <strong>X</strong> (<strong>X</strong> <strong>X</strong>')^-1 (<strong>X</strong>' <strong>z</strong> + 1/2 <strong>v</strong>) using <strong>P</strong> <strong>z</strong>, where <strong>P</strong> is a projection matrix satisfying <strong>P</strong>' = <strong>P</strong> and <strong>P</strong>^2 = <strong>P</strong>? How can I estimate such <strong>P</strong>?</p>

<p>Thanks in advance!</p>