Identifying architectural design decisions for achieving green ML
  serving
Francisco Durán Silverio Martínez-Fernández Matias Martinez Patricia Lago
The growing use of large machine learning models highlights concerns about
their increasing computational demands. While the energy consumption of their
training phase has received attention, fewer works have considered the
inference phase. For ML inference, the binding of ML models to the ML system
for user access, known as ML serving, is a critical yet understudied step for
achieving efficiency in ML applications.
  We examine the literature in ML architectural design decisions and Green AI,
with a special focus on ML serving. The aim is to analyze ML serving
architectural design decisions for the purpose of understanding and identifying
them with respect to quality characteristics from the point of view of
researchers and practitioners in the context of ML serving literature.
  Our results (i) identify ML serving architectural design decisions along with
their corresponding components and associated technological stack, and (ii)
provide an overview of the quality characteristics studied in the literature,
including energy efficiency.
  This preliminary study is the first step in our goal to achieve green ML
serving. Our analysis may aid ML researchers and practitioners in making
green-aware architecture design decisions when serving their models.