Spatio-Temporal Surrogates for Interaction of a Jet with High
  Explosives: Part II -- Clustering Extremely High-Dimensional Grid-Based Data
Chandrika Kamath Juliette S. Franzman
Building an accurate surrogate model for the spatio-temporal outputs of a
computer simulation is a challenging task. A simple approach to improve the
accuracy of the surrogate is to cluster the outputs based on similarity and
build a separate surrogate model for each cluster. This clustering is
relatively straightforward when the output at each time step is of moderate
size. However, when the spatial domain is represented by a large number of grid
points, numbering in the millions, the clustering of the data becomes more
challenging. In this report, we consider output data from simulations of a jet
interacting with high explosives. These data are available on spatial domains
of different sizes, at grid points that vary in their spatial coordinates, and
in a format that distributes the output across multiple files at each time step
of the simulation. We first describe how we bring these data into a consistent
format prior to clustering. Borrowing the idea of random projections from data
mining, we reduce the dimension of our data by a factor of thousand, making it
possible to use the iterative k-means method for clustering. We show how we can
use the randomness of both the random projections, and the choice of initial
centroids in k-means clustering, to determine the number of clusters in our
data set. Our approach makes clustering of extremely high dimensional data
tractable, generating meaningful cluster assignments for our problem, despite
the approximation introduced in the random projections.