D2NO: Efficient Handling of Heterogeneous Input Function Spaces with
  Distributed Deep Neural Operators
Zecheng Zhang Christian Moya Lu Lu Guang Lin Hayden Schaeffer
Neural operators have been applied in various scientific fields, such as
solving parametric partial differential equations, dynamical systems with
control, and inverse problems. However, challenges arise when dealing with
input functions that exhibit heterogeneous properties, requiring multiple
sensors to handle functions with minimal regularity. To address this issue,
discretization-invariant neural operators have been used, allowing the sampling
of diverse input functions with different sensor locations. However, existing
frameworks still require an equal number of sensors for all functions. In our
study, we propose a novel distributed approach to further relax the
discretization requirements and solve the heterogeneous dataset challenges. Our
method involves partitioning the input function space and processing individual
input functions using independent and separate neural networks. A centralized
neural network is used to handle shared information across all output
functions. This distributed methodology reduces the number of gradient descent
back-propagation steps, improving efficiency while maintaining accuracy. We
demonstrate that the corresponding neural network is a universal approximator
of continuous nonlinear operators and present four numerical examples to
validate its performance.