 The storage requirements at every step formula_31 here are to store the matrix formula_59, which is constant at formula_39.For the case when formula_44 is not invertible, consider the regularised version of the problem 
loss function formula_62.Then, it's easy to show that the same algorithm works with formula_63, and the iterations proceed to give formula_64.