Toward Sustainable GenAI using Generation Directives for Carbon-Friendly
  Large Language Model Inference
Baolin Li Yankai Jiang Vijay Gadepally Devesh Tiwari
The rapid advancement of Generative Artificial Intelligence (GenAI) across
diverse sectors raises significant environmental concerns, notably the carbon
emissions from their cloud and high performance computing (HPC) infrastructure.
This paper presents Sprout, an innovative framework designed to address these
concerns by reducing the carbon footprint of generative Large Language Model
(LLM) inference services. Sprout leverages the innovative concept of
"generation directives" to guide the autoregressive generation process, thereby
enhancing carbon efficiency. Our proposed method meticulously balances the need
for ecological sustainability with the demand for high-quality generation
outcomes. Employing a directive optimizer for the strategic assignment of
generation directives to user prompts and an original offline quality
evaluator, Sprout demonstrates a significant reduction in carbon emissions by
over 40% in real-world evaluations using the Llama2 LLM and global electricity
grid data. This research marks a critical step toward aligning AI technology
with sustainable practices, highlighting the potential for mitigating
environmental impacts in the rapidly expanding domain of generative artificial
intelligence.