Local Risk Bounds for Statistical Aggregation
Jaouad Mourtada Tomas Vaškevičius Nikita Zhivotovskiy
In the problem of aggregation, the aim is to combine a given class of base
predictors to achieve predictions nearly as accurate as the best one. In this
flexible framework, no assumption is made on the structure of the class or the
nature of the target. Aggregation has been studied in both sequential and
statistical contexts. Despite some important differences between the two
problems, the classical results in both cases feature the same global
complexity measure. In this paper, we revisit and tighten classical results in
the theory of aggregation in the statistical setting by replacing the global
complexity with a smaller, local one. Some of our proofs build on the PAC-Bayes
localization technique introduced by Catoni. Among other results, we prove
localized versions of the classical bound for the exponential weights estimator
due to Leung and Barron and deviation-optimal bounds for the Q-aggregation
estimator. These bounds improve over the results of Dai, Rigollet and Zhang for
fixed design regression and the results of Lecu\'e and Rigollet for random
design regression.