Kolmogorov n-Widths for Multitask Physics-Informed Machine Learning
  (PIML) Methods: Towards Robust Metrics
Michael Penwarden Houman Owhadi Robert M. Kirby
Physics-informed machine learning (PIML) as a means of solving partial
differential equations (PDE) has garnered much attention in the Computational
Science and Engineering (CS&E) world. This topic encompasses a broad array of
methods and models aimed at solving a single or a collection of PDE problems,
called multitask learning. PIML is characterized by the incorporation of
physical laws into the training process of machine learning models in lieu of
large data when solving PDE problems. Despite the overall success of this
collection of methods, it remains incredibly difficult to analyze, benchmark,
and generally compare one approach to another. Using Kolmogorov n-widths as a
measure of effectiveness of approximating functions, we judiciously apply this
metric in the comparison of various multitask PIML architectures. We compute
lower accuracy bounds and analyze the model's learned basis functions on
various PDE problems. This is the first objective metric for comparing
multitask PIML architectures and helps remove uncertainty in model validation
from selective sampling and overfitting. We also identify avenues of
improvement for model architectures, such as the choice of activation function,
which can drastically affect model generalization to "worst-case" scenarios,
which is not observed when reporting task-specific errors. We also incorporate
this metric into the optimization process through regularization, which
improves the models' generalizability over the multitask PDE problem.