Engineering software 2.0 by interpolating neural networks: unifying
  training, solving, and calibration
Chanwook Park Sourav Saha Jiachen Guo Xiaoyu Xie Satyajit Mojumder Miguel A. Bessa Dong Qian Wei Chen Gregory J. Wagner Jian Cao Wing Kam Liu
The evolution of artificial intelligence (AI) and neural network theories has
revolutionized the way software is programmed, shifting from a hard-coded
series of codes to a vast neural network. However, this transition in
engineering software has faced challenges such as data scarcity, multi-modality
of data, low model accuracy, and slow inference. Here, we propose a new network
based on interpolation theories and tensor decomposition, the interpolating
neural network (INN). Instead of interpolating training data, a common notion
in computer science, INN interpolates interpolation points in the physical
space whose coordinates and values are trainable. It can also extrapolate if
the interpolation points reside outside of the range of training data and the
interpolation functions have a larger support domain. INN features orders of
magnitude fewer trainable parameters, faster training, a smaller memory
footprint, and higher model accuracy compared to feed-forward neural networks
(FFNN) or physics-informed neural networks (PINN). INN is poised to usher in
Engineering Software 2.0, a unified neural network that spans various domains
of space, time, parameters, and initial/boundary conditions. This has
previously been computationally prohibitive due to the exponentially growing
number of trainable parameters, easily exceeding the parameter size of ChatGPT,
which is over 1 trillion. INN addresses this challenge by leveraging tensor
decomposition and tensor product, with adaptable network architecture.