TFBEST: Dual-Aspect Transformer with Learnable Positional Encoding for
  Failure Prediction
Rohan Mohapatra Saptarshi Sengupta
Hard Disk Drive (HDD) failures in datacenters are costly - from catastrophic
data loss to a question of goodwill, stakeholders want to avoid it like the
plague. An important tool in proactively monitoring against HDD failure is
timely estimation of the Remaining Useful Life (RUL). To this end, the
Self-Monitoring, Analysis and Reporting Technology employed within HDDs
(S.M.A.R.T.) provide critical logs for long-term maintenance of the security
and dependability of these essential data storage devices. Data-driven
predictive models in the past have used these S.M.A.R.T. logs and CNN/RNN based
architectures heavily. However, they have suffered significantly in providing a
confidence interval around the predicted RUL values as well as in processing
very long sequences of logs. In addition, some of these approaches, such as
those based on LSTMs, are inherently slow to train and have tedious feature
engineering overheads. To overcome these challenges, in this work we propose a
novel transformer architecture - a Temporal-fusion Bi-encoder Self-attention
Transformer (TFBEST) for predicting failures in hard-drives. It is an
encoder-decoder based deep learning technique that enhances the context gained
from understanding health statistics sequences and predicts a sequence of the
number of days remaining before a disk potentially fails. In this paper, we
also provide a novel confidence margin statistic that can help manufacturers
replace a hard-drive within a time frame. Experiments on Seagate HDD data show
that our method significantly outperforms the state-of-the-art RUL prediction
methods during testing over the exhaustive 10-year data from Backblaze
(2013-present). Although validated on HDD failure prediction, the TFBEST
architecture is well-suited for other prognostics applications and may be
adapted for allied regression problems.