Comparing AutoML and Deep Learning Methods for Condition Monitoring
  using Realistic Validation Scenarios
Payman Goodarzi Andreas Schütze Tizian Schneider
This study extensively compares conventional machine learning methods and
deep learning for condition monitoring tasks using an AutoML toolbox. The
experiments reveal consistent high accuracy in random K-fold cross-validation
scenarios across all tested models. However, when employing leave-one-group-out
(LOGO) cross-validation on the same datasets, no clear winner emerges,
indicating the presence of domain shift in real-world scenarios. Additionally,
the study assesses the scalability and interpretability of conventional methods
and neural networks. Conventional methods offer explainability with their
modular structure aiding feature identification. In contrast, neural networks
require specialized interpretation techniques like occlusion maps to visualize
important regions in the input data. Finally, the paper highlights the
significance of feature selection, particularly in condition monitoring tasks
with limited class variations. Low-complexity models prove sufficient for such
tasks, as only a few features from the input signal are typically needed. In
summary, these findings offer crucial insights into the strengths and
limitations of various approaches, providing valuable benchmarks and
identifying the most suitable methods for condition monitoring applications,
thereby enhancing their applicability in real-world scenarios.