Backpropagation in Gradient Descent for Neural Networks vs. Linear Regression
<p>I'm trying to understand "Back Propagation" as it is used in Neural Nets that are optimized using Gradient Descent. Reading through the literature it seems to do a few things.</p>

<ol>
<li>Use random weights to start with and get error values</li>
<li>Perform Gradient Descent on the loss function using these weights to arrive at new weights.</li>
<li>Update the weights with these new weights until the loss function is minimized.</li>
</ol>

<p>The steps above seem to be the EXACT process to solve for Linear Models (Regression for e.g.)? Andrew Ng's excellent course on Coursera for Machine Learning does exactly that for Linear Regression. </p>

<p>So, I'm trying to understand if BackPropagation does anything more than gradient descent on the loss function.. and if not, why is it only referenced in the case of Neural Nets and why not for GLMs (Generalized Linear Models). They all seem to be doing the same thing- what might I be missing?</p>