Novel Representation Learning Technique using Graphs for Performance
  Analytics
Tarek Ramadan Ankur Lahiry Tanzima Z. Islam
The performance analytics domain in High Performance Computing (HPC) uses
tabular data to solve regression problems, such as predicting the execution
time. Existing Machine Learning (ML) techniques leverage the correlations among
features given tabular datasets, not leveraging the relationships between
samples directly. Moreover, since high-quality embeddings from raw features
improve the fidelity of the downstream predictive models, existing methods rely
on extensive feature engineering and pre-processing steps, costing time and
manual effort. To fill these two gaps, we propose a novel idea of transforming
tabular performance data into graphs to leverage the advancement of Graph
Neural Network-based (GNN) techniques in capturing complex relationships
between features and samples. In contrast to other ML application domains, such
as social networks, the graph is not given; instead, we need to build it. To
address this gap, we propose graph-building methods where nodes represent
samples, and the edges are automatically inferred iteratively based on the
similarity between the features in the samples. We evaluate the effectiveness
of the generated embeddings from GNNs based on how well they make even a simple
feed-forward neural network perform for regression tasks compared to other
state-of-the-art representation learning techniques. Our evaluation
demonstrates that even with up to 25% random missing values for each dataset,
our method outperforms commonly used graph and Deep Neural Network (DNN)-based
approaches and achieves up to 61.67% & 78.56% improvement in MSE loss over the
DNN baseline respectively for HPC dataset and Machine Learning Datasets.