A Duality Analysis of Kernel Ridge Regression in the Noiseless Regime
Jihao Long Xiaojun Peng Lei Wu
In this paper, we conduct a comprehensive analysis of generalization
properties of Kernel Ridge Regression (KRR) in the noiseless regime, a scenario
crucial to scientific computing, where data are often generated via computer
simulations. We prove that KRR can attain the minimax optimal rate, which
depends on both the eigenvalue decay of the associated kernel and the relative
smoothness of target functions. Particularly, when the eigenvalue decays
exponentially fast, KRR achieves the spectral accuracy, i.e., a convergence
rate faster than any polynomial. Moreover, the numerical experiments well
corroborate our theoretical findings. Our proof leverages a novel extension of
the duality framework introduced by Chen et al. (2023), which could be useful
in analyzing kernel-based methods beyond the scope of this work.