Measuring the Energy Consumption and Efficiency of Deep Neural Networks:
  An Empirical Analysis and Design Recommendations
Charles Edison Tripp Jordan Perr-Sauer Jamil Gafur Amabarish Nag Avi Purkayastha Sagi Zisman Erik A. Bensen
Addressing the so-called ``Red-AI'' trend of rising energy consumption by
large-scale neural networks, this study investigates the actual energy
consumption, as measured by node-level watt-meters, of training various fully
connected neural network architectures. We introduce the BUTTER-E dataset, an
augmentation to the BUTTER Empirical Deep Learning dataset, containing energy
consumption and performance data from 63,527 individual experimental runs
spanning 30,582 distinct configurations: 13 datasets, 20 sizes (number of
trainable parameters), 8 network ``shapes'', and 14 depths on both CPU and GPU
hardware collected using node-level watt-meters. This dataset reveals the
complex relationship between dataset size, network structure, and energy use,
and highlights the impact of cache effects. We propose a straightforward and
effective energy model that accounts for network size, computing, and memory
hierarchy. Our analysis also uncovers a surprising, hardware-mediated
non-linear relationship between energy efficiency and network design,
challenging the assumption that reducing the number of parameters or FLOPs is
the best way to achieve greater energy efficiency. Highlighting the need for
cache-considerate algorithm development, we suggest a combined approach to
energy efficient network, algorithm, and hardware design. This work contributes
to the fields of sustainable computing and Green AI, offering practical
guidance for creating more energy-efficient neural networks and promoting
sustainable AI.