Attention-enhanced neural differential equations for physics-informed
  deep learning of ion transport
Danyal Rehman John H. Lienhard
Species transport models typically combine partial differential equations
(PDEs) with relations from hindered transport theory to quantify
electromigrative, convective, and diffusive transport through complex
nanoporous systems; however, these formulations are frequently substantial
simplifications of the governing dynamics, leading to the poor generalization
performance of PDE-based models. Given the growing interest in deep learning
methods for the physical sciences, we develop a machine learning-based approach
to characterize ion transport across nanoporous membranes. Our proposed
framework centers around attention-enhanced neural differential equations that
incorporate electroneutrality-based inductive biases to improve generalization
performance relative to conventional PDE-based methods. In addition, we study
the role of the attention mechanism in illuminating physically-meaningful
ion-pairing relationships across diverse mixture compositions. Further, we
investigate the importance of pre-training on simulated data from PDE-based
models, as well as the performance benefits from hard vs. soft inductive
biases. Our results indicate that physics-informed deep learning solutions can
outperform their classical PDE-based counterparts and provide promising avenues
for modelling complex transport phenomena across diverse applications.