Exploring Winograd Convolution for Cost-effective Neural Network Fault
  Tolerance
Xinghua Xue Cheng Liu Bo Liu Haitong Huang Ying Wang Tao Luo Lei Zhang Huawei Li Xiaowei Li
Winograd is generally utilized to optimize convolution performance and
computational efficiency because of the reduced multiplication operations, but
the reliability issues brought by winograd are usually overlooked. In this
work, we observe the great potential of winograd convolution in improving
neural network (NN) fault tolerance. Based on the observation, we evaluate
winograd convolution fault tolerance comprehensively from different
granularities ranging from models, layers, and operation types for the first
time. Then, we explore the use of inherent fault tolerance of winograd
convolution for cost-effective NN protection against soft errors. Specifically,
we mainly investigate how winograd convolution can be effectively incorporated
with classical fault-tolerant design approaches including triple modular
redundancy (TMR), fault-aware retraining, and constrained activation functions.
According to our experiments, winograd convolution can reduce the
fault-tolerant design overhead by 55.77\% on average without any accuracy loss
compared to standard convolution, and further reduce the computing overhead by
17.24\% when the inherent fault tolerance of winograd convolution is
considered. When it is applied on fault-tolerant neural networks enhanced with
fault-aware retraining and constrained activation functions, the resulting
model accuracy generally shows significant improvement in presence of various
faults.