ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A
  Case Study
Hala Abdelkader Mohamed Abdelrazek Scott Barnett Jean-Guy Schneider Priya Rani Rajesh Vasa
Machine learning (ML), especially with the emergence of large language models
(LLMs), has significantly transformed various industries. However, the
transition from ML model prototyping to production use within software systems
presents several challenges. These challenges primarily revolve around ensuring
safety, security, and transparency, subsequently influencing the overall
robustness and trustworthiness of ML models. In this paper, we introduce
ML-On-Rails, a protocol designed to safeguard ML models, establish a
well-defined endpoint interface for different ML tasks, and clear communication
between ML providers and ML consumers (software engineers). ML-On-Rails
enhances the robustness of ML models via incorporating detection capabilities
to identify unique challenges specific to production ML. We evaluated the
ML-On-Rails protocol through a real-world case study of the MoveReminder
application. Through this evaluation, we emphasize the importance of
safeguarding ML models in production.