Extending Variability-Aware Model Selection with Bias Detection in
  Machine Learning Projects
Cristina Tavares Nathalia Nascimento Paulo Alencar Donald Cowan
Data science projects often involve various machine learning (ML) methods
that depend on data, code, and models. One of the key activities in these
projects is the selection of a model or algorithm that is appropriate for the
data analysis at hand. ML model selection depends on several factors, which
include data-related attributes such as sample size, functional requirements
such as the prediction algorithm type, and non-functional requirements such as
performance and bias. However, the factors that influence such selection are
often not well understood and explicitly represented. This paper describes
ongoing work on extending an adaptive variability-aware model selection method
with bias detection in ML projects. The method involves: (i) modeling the
variability of the factors that affect model selection using feature models
based on heuristics proposed in the literature; (ii) instantiating our
variability model with added features related to bias (e.g., bias-related
metrics); and (iii) conducting experiments that illustrate the method in a
specific case study to illustrate our approach based on a heart failure
prediction project. The proposed approach aims to advance the state of the art
by making explicit factors that influence model selection, particularly those
related to bias, as well as their interactions. The provided representations
can transform model selection in ML projects into a non ad hoc, adaptive, and
explainable process.