A PAC-Bayesian Link Between Generalisation and Flat Minima
Maxime Haddouche Paul Viallard Umut Simsekli Benjamin Guedj
Modern machine learning usually involves predictors in the overparametrised
setting (number of trained parameters greater than dataset size), and their
training yield not only good performances on training data, but also good
generalisation capacity. This phenomenon challenges many theoretical results,
and remains an open problem. To reach a better understanding, we provide novel
generalisation bounds involving gradient terms. To do so, we combine the
PAC-Bayes toolbox with Poincar\'e and Log-Sobolev inequalities, avoiding an
explicit dependency on dimension of the predictor space. Our results highlight
the positive influence of \emph{flat minima} (being minima with a neighbourhood
nearly minimising the learning problem as well) on generalisation performances,
involving directly the benefits of the optimisation phase.