 When this 
is replaced by
or formula_67 by formula_68, this becomes the stochastic gradient descent algorithm.In this case, the complexity for formula_41 steps of this algorithm reduces to formula_70.The storage requirements at every step formula_31 are constant at formula_72.