Managing ML-Based Application Non-Functional Behavior: A Multi-Model
  Approach
Marco Anisetti Claudio A. Ardagna Nicola Bena Ernesto Damiani Paolo G. Panero
Modern applications are increasingly driven by Machine Learning (ML) models
whose non-deterministic behavior is affecting the entire application life cycle
from design to operation. The pervasive adoption of ML is urgently calling for
approaches that guarantee a stable non-functional behavior of ML-based
applications over time and across model changes. To this aim, non-functional
properties of ML models, such as privacy, confidentiality, fairness, and
explainability, must be monitored, verified, and maintained. This need is even
more pressing when modern applications operate in the edge-cloud continuum,
increasing their complexity and dynamicity. Existing approaches mostly focus on
i) implementing classifier selection solutions according to the functional
behavior of ML models, ii) finding new algorithmic solutions to this need, such
as continuous re-training. In this paper, we propose a multi-model approach
built on dynamic classifier selection, where multiple ML models showing similar
non-functional properties are made available to the application and one model
is selected over time according to (dynamic and unpredictable) contextual
changes. Our solution goes beyond the state of the art by providing an
architectural and methodological approach that continuously guarantees a stable
non-functional behavior of ML-based applications, is applicable to different ML
models, and is driven by non-functional properties assessed on the models
themselves. It consists of a two-step process working during application
operation, where model assessment verifies non-functional properties of ML
models trained and selected at development time, and model substitution
guarantees a continuous and stable support of non-functional properties. We
experimentally evaluate our solution in a real-world scenario focusing on
non-functional property fairness.