Deciphering AutoML Ensembles: cattleia's Assistance in Decision-Making
Anna Kozak Dominik Kędzierski Jakub Piwko Malwina Wojewoda Katarzyna Woźnica
In many applications, model ensembling proves to be better than a single
predictive model. Hence, it is the most common post-processing technique in
Automated Machine Learning (AutoML). The most popular frameworks use ensembles
at the expense of reducing the interpretability of the final models. In our
work, we propose cattleia - an application that deciphers the ensembles for
regression, multiclass, and binary classification tasks. This tool works with
models built by three AutoML packages: auto-sklearn, AutoGluon, and FLAML. The
given ensemble is analyzed from different perspectives. We conduct a predictive
performance investigation through evaluation metrics of the ensemble and its
component models. We extend the validation perspective by introducing new
measures to assess the diversity and complementarity of the model predictions.
Moreover, we apply explainable artificial intelligence (XAI) techniques to
examine the importance of variables. Summarizing obtained insights, we can
investigate and adjust the weights with a modification tool to tune the
ensemble in the desired way. The application provides the aforementioned
aspects through dedicated interactive visualizations, making it accessible to a
diverse audience. We believe the cattleia can support users in decision-making
and deepen the comprehension of AutoML frameworks.