Online Job Failure Prediction in an HPC System
Francesco Antici Andrea Borghesi Zeynep Kiziltan
Modern High Performance Computing (HPC) systems are complex machines, with
major impacts on economy and society. Along with their computational
capability, their energy consumption is also steadily raising, representing a
critical issue given the ongoing environmental and energetic crisis. Therefore,
developing strategies to optimize HPC system management has paramount
importance, both to guarantee top-tier performance and to improve energy
efficiency. One strategy is to act at the workload level and highlight the jobs
that are most likely to fail, prior to their execution on the system. Jobs
failing during their execution unnecessarily occupy resources which could delay
other jobs, adversely affecting the system performance and energy consumption.
In this paper, we study job failure prediction at submit-time using classical
machine learning algorithms. Our novelty lies in (i) the combination of these
algorithms with Natural Language Processing (NLP) tools to represent jobs and
(ii) the design of the approach to work in an online fashion in a real system.
The study is based on a dataset extracted from a production machine hosted at
the HPC centre CINECA in Italy. Experimental results show that our approach is
promising.