Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation
Sobihan Surendran Antoine Godichon-Baggioni Adeline Fermanian Sylvain Le Corff
Stochastic Gradient Descent (SGD) with adaptive steps is now widely used for
training deep neural networks. Most theoretical results assume access to
unbiased gradient estimators, which is not the case in several recent deep
learning and reinforcement learning applications that use Monte Carlo methods.
This paper provides a comprehensive non-asymptotic analysis of SGD with biased
gradients and adaptive steps for convex and non-convex smooth functions. Our
study incorporates time-dependent bias and emphasizes the importance of
controlling the bias and Mean Squared Error (MSE) of the gradient estimator. In
particular, we establish that Adagrad and RMSProp with biased gradients
converge to critical points for smooth non-convex functions at a rate similar
to existing results in the literature for the unbiased case. Finally, we
provide experimental results using Variational Autoenconders (VAE) that
illustrate our convergence results and show how the effect of bias can be
reduced by appropriate hyperparameter tuning.