On the convergence of the MLE as an estimator of the learning rate in
  the Exp3 algorithm
Julien Aubert Luc Lehéricy Patricia Reynaud-Bouret
When fitting the learning data of an individual to algorithm-like learning
models, the observations are so dependent and non-stationary that one may
wonder what the classical Maximum Likelihood Estimator (MLE) could do, even if
it is the usual tool applied to experimental cognition. Our objective in this
work is to show that the estimation of the learning rate cannot be efficient if
the learning rate is constant in the classical Exp3 (Exponential weights for
Exploration and Exploitation) algorithm. Secondly, we show that if the learning
rate decreases polynomially with the sample size, then the prediction error and
in some cases the estimation error of the MLE satisfy bounds in probability
that decrease at a polynomial rate.