Learning Theory of Distribution Regression with Neural Networks
Zhongjie Shi Zhan Yu Ding-Xuan Zhou
In this paper, we aim at establishing an approximation theory and a learning
theory of distribution regression via a fully connected neural network (FNN).
In contrast to the classical regression methods, the input variables of
distribution regression are probability measures. Then we often need to perform
a second-stage sampling process to approximate the actual information of the
distribution. On the other hand, the classical neural network structure
requires the input variable to be a vector. When the input samples are
probability distributions, the traditional deep neural network method cannot be
directly used and the difficulty arises for distribution regression. A
well-defined neural network structure for distribution inputs is intensively
desirable. There is no mathematical model and theoretical analysis on neural
network realization of distribution regression. To overcome technical
difficulties and address this issue, we establish a novel fully connected
neural network framework to realize an approximation theory of functionals
defined on the space of Borel probability measures. Furthermore, based on the
established functional approximation results, in the hypothesis space induced
by the novel FNN structure with distribution inputs, almost optimal learning
rates for the proposed distribution regression model up to logarithmic terms
are derived via a novel two-stage error decomposition technique.