Toward Cross-Layer Energy Optimizations in Machine Learning Systems
Jae-Won Chung Mosharaf Chowdhury
The enormous energy consumption of machine learning (ML) and generative AI
workloads shows no sign of waning, taking a toll on operating costs, power
delivery, and environmental sustainability. Despite a long line of research on
energy-efficient hardware, we found that software plays a critical role in ML
energy optimization through two recent works: Zeus and Perseus. This is
especially true for large language models (LLMs) because their model sizes and,
therefore, energy demands are growing faster than hardware efficiency
improvements. Therefore, we advocate for a cross-layer approach for energy
optimizations in ML systems, where hardware provides architectural support that
pushes energy-efficient software further, while software leverages and
abstracts the hardware to develop techniques that bring hardware-agnostic
energy-efficiency gains.