 When combined with backpropagation, this is currently the de facto training method for training artificial neural networks.The simple example of linear least squares is used to explain a variety of ideas in online learning.The ideas are general enough to be applied to other settings, for example, with other convex loss functions.