A practical existence theorem for reduced order models based on
  convolutional autoencoders
Nicola Rares Franco Simone Brugiapaglia
In recent years, deep learning has gained increasing popularity in the fields
of Partial Differential Equations (PDEs) and Reduced Order Modeling (ROM),
providing domain practitioners with new powerful data-driven techniques such as
Physics-Informed Neural Networks (PINNs), Neural Operators, Deep Operator
Networks (DeepONets) and Deep-Learning based ROMs (DL-ROMs). In this context,
deep autoencoders based on Convolutional Neural Networks (CNNs) have proven
extremely effective, outperforming established techniques, such as the reduced
basis method, when dealing with complex nonlinear problems. However, despite
the empirical success of CNN-based autoencoders, there are only a few
theoretical results supporting these architectures, usually stated in the form
of universal approximation theorems. In particular, although the existing
literature provides users with guidelines for designing convolutional
autoencoders, the subsequent challenge of learning the latent features has been
barely investigated. Furthermore, many practical questions remain unanswered,
e.g., the number of snapshots needed for convergence or the neural network
training strategy. In this work, using recent techniques from sparse
high-dimensional function approximation, we fill some of these gaps by
providing a new practical existence theorem for CNN-based autoencoders when the
parameter-to-solution map is holomorphic. This regularity assumption arises in
many relevant classes of parametric PDEs, such as the parametric diffusion
equation, for which we discuss an explicit application of our general theory.