Linear bandits with polylogarithmic minimax regret
Josep Lumbreras Marco Tomamichel
We study a noise model for linear stochastic bandits for which the
subgaussian noise parameter vanishes linearly as we select actions on the unit
sphere closer and closer to the unknown vector. We introduce an algorithm for
this problem that exhibits a minimax regret scaling as $\log^3(T)$ in the time
horizon $T$, in stark contrast the square root scaling of this regret for
typical bandit algorithms. Our strategy, based on weighted least-squares
estimation, achieves the eigenvalue relation $\lambda_{\min} ( V_t ) = \Omega
(\sqrt{\lambda_{\max}(V_t ) })$ for the design matrix $V_t$ at each time step
$t$ through geometrical arguments that are independent of the noise model and
might be of independent interest. This allows us to tightly control the
expected regret in each time step to be of the order $O(\frac1{t})$, leading to
the logarithmic scaling of the cumulative regret.