Tighter Generalisation Bounds via Interpolation
Paul Viallard Maxime Haddouche Umut Şimşekli Benjamin Guedj
This paper contains a recipe for deriving new PAC-Bayes generalisation bounds
based on the $(f, \Gamma)$-divergence, and, in addition, presents PAC-Bayes
generalisation bounds where we interpolate between a series of probability
divergences (including but not limited to KL, Wasserstein, and total
variation), making the best out of many worlds depending on the posterior
distributions properties. We explore the tightness of these bounds and connect
them to earlier results from statistical learning, which are specific cases. We
also instantiate our bounds as training objectives, yielding non-trivial
guarantees and practical performances.