Towards Self-Adaptive Machine Learning-Enabled Systems Through QoS-Aware
  Model Switching
Shubham Kulkarni Arya Marda Karthik Vaidhyanathan
Machine Learning (ML), particularly deep learning, has seen vast
advancements, leading to the rise of Machine Learning-Enabled Systems (MLS).
However, numerous software engineering challenges persist in propelling these
MLS into production, largely due to various run-time uncertainties that impact
the overall Quality of Service (QoS). These uncertainties emanate from ML
models, software components, and environmental factors. Self-adaptation
techniques present potential in managing run-time uncertainties, but their
application in MLS remains largely unexplored. As a solution, we propose the
concept of a Machine Learning Model Balancer, focusing on managing
uncertainties related to ML models by using multiple models. Subsequently, we
introduce AdaMLS, a novel self-adaptation approach that leverages this concept
and extends the traditional MAPE-K loop for continuous MLS adaptation. AdaMLS
employs lightweight unsupervised learning for dynamic model switching, thereby
ensuring consistent QoS. Through a self-adaptive object detection system
prototype, we demonstrate AdaMLS's effectiveness in balancing system and model
performance. Preliminary results suggest AdaMLS surpasses naive and single
state-of-the-art models in QoS guarantees, heralding the advancement towards
self-adaptive MLS with optimal QoS in dynamic environments.