Information theory unifies atomistic machine learning, uncertainty
  quantification, and materials thermodynamics
Daniel Schwalbe-Koda Sebastien Hamel Babak Sadigh Fei Zhou Vincenzo Lordi
An accurate description of information is relevant for a range of problems in
atomistic modeling, such as sampling methods, detecting rare events, analyzing
datasets, or performing uncertainty quantification (UQ) in machine learning
(ML)-driven simulations. Although individual methods have been proposed for
each of these tasks, they lack a common theoretical background integrating
their solutions. Here, we introduce an information theoretical framework that
unifies predictions of phase transformations, kinetic events, dataset
optimality, and model-free UQ from atomistic simulations, thus bridging
materials modeling, ML, and statistical mechanics. We first demonstrate that,
for a proposed representation, the information entropy of a distribution of
atom-centered environments is a surrogate value for thermodynamic entropy.
Using molecular dynamics (MD) simulations, we show that information entropy
differences from trajectories can be used to build phase diagrams, identify
rare events, and recover classical theories of nucleation. Building on these
results, we use this general concept of entropy to quantify information in
datasets for ML interatomic potentials (IPs), informing compression, explaining
trends in testing errors, and evaluating the efficiency of active learning
strategies. Finally, we propose a model-free UQ method for MLIPs using
information entropy, showing it reliably detects extrapolation regimes, scales
to millions of atoms, and goes beyond model errors. This method is made
available as the package QUESTS: Quick Uncertainty and Entropy via STructural
Similarity, providing a new unifying theory for data-driven atomistic modeling
and combining efforts in ML, first-principles thermodynamics, and simulations.