Watt For What: Rethinking Deep Learning's Energy-Performance
  Relationship
Shreyank N Gowda Xinyue Hao Gen Li Laura Sevilla-Lara Shashank Narayana Gowda
Deep learning models have revolutionized various fields, from image
recognition to natural language processing, by achieving unprecedented levels
of accuracy. However, their increasing energy consumption has raised concerns
about their environmental impact, disadvantaging smaller entities in research
and exacerbating global energy consumption. In this paper, we explore the
trade-off between model accuracy and electricity consumption, proposing a
metric that penalizes large consumption of electricity. We conduct a
comprehensive study on the electricity consumption of various deep learning
models across different GPUs, presenting a detailed analysis of their
accuracy-efficiency trade-offs. By evaluating accuracy per unit of electricity
consumed, we demonstrate how smaller, more energy-efficient models can
significantly expedite research while mitigating environmental concerns. Our
results highlight the potential for a more sustainable approach to deep
learning, emphasizing the importance of optimizing models for efficiency. This
research also contributes to a more equitable research landscape, where smaller
entities can compete effectively with larger counterparts. This advocates for
the adoption of efficient deep learning practices to reduce electricity
consumption, safeguarding the environment for future generations whilst also
helping ensure a fairer competitive landscape.