Neural network scoring for efficient computing
Hugo Waltsburger Erwan Libessart Chengfang Ren Anthony Kolar Regis Guinvarc'h
Much work has been dedicated to estimating and optimizing workloads in
high-performance computing (HPC) and deep learning. However, researchers have
typically relied on few metrics to assess the efficiency of those techniques.
Most notably, the accuracy, the loss of the prediction, and the computational
time with regard to GPUs or/and CPUs characteristics. It is rare to see figures
for power consumption, partly due to the difficulty of obtaining accurate power
readings. In this paper, we introduce a composite score that aims to
characterize the trade-off between accuracy and power consumption measured
during the inference of neural networks. For this purpose, we present a new
open-source tool allowing researchers to consider more metrics: granular power
consumption, but also RAM/CPU/GPU utilization, as well as storage, and network
input/output (I/O). To our best knowledge, it is the first fit test for neural
architectures on hardware architectures. This is made possible thanks to
reproducible power efficiency measurements. We applied this procedure to
state-of-the-art neural network architectures on miscellaneous hardware. One of
the main applications and novelties is the measurement of algorithmic power
efficiency. The objective is to allow researchers to grasp their algorithms'
efficiencies better. This methodology was developed to explore trade-offs
between energy usage and accuracy in neural networks. It is also useful when
fitting hardware for a specific task or to compare two architectures more
accurately, with architecture exploration in mind.