Masked Transformer for Electrocardiogram Classification
Ya Zhou Xiaolin Diao Yanni Huo Yang Liu Xiaohan Fan Wei Zhao
Electrocardiogram (ECG) is one of the most important diagnostic tools in
clinical applications. With the advent of advanced algorithms, various deep
learning models have been adopted for ECG tasks. However, the potential of
Transformers for ECG data is not yet realized, despite their widespread success
in computer vision and natural language processing. In this work, we present a
useful masked Transformer method for ECG classification referred to as MTECG,
which expands the application of masked autoencoders to ECG time series. We
construct a dataset comprising 220,251 ECG recordings with a broad range of
diagnoses annoated by medical experts to explore the properties of MTECG. Under
the proposed training strategies, a lightweight model with 5.7M parameters
performs stably well on a broad range of masking ratios (5%-75%). The ablation
studies highlight the importance of fluctuated reconstruction targets, training
schedule length, layer-wise LR decay and DropPath rate. The experiments on both
private and public ECG datasets demonstrate that MTECG-T significantly
outperforms the recent state-of-the-art algorithms in ECG classification.