Dynamic ensemble selection based on Deep Neural Network Uncertainty
  Estimation for Adversarial Robustness
Ruoxi Qin Linyuan Wang Xuehui Du Xingyuan Chen Bin Yan
The deep neural network has attained significant efficiency in image
recognition. However, it has vulnerable recognition robustness under extensive
data uncertainty in practical applications. The uncertainty is attributed to
the inevitable ambient noise and, more importantly, the possible adversarial
attack. Dynamic methods can effectively improve the defense initiative in the
arms race of attack and defense of adversarial examples. Different from the
previous dynamic method depend on input or decision, this work explore the
dynamic attributes in model level through dynamic ensemble selection technology
to further protect the model from white-box attacks and improve the robustness.
Specifically, in training phase the Dirichlet distribution is apply as prior of
sub-models' predictive distribution, and the diversity constraint in parameter
space is introduced under the lightweight sub-models to construct alternative
ensembel model spaces. In test phase, the certain sub-models are dynamically
selected based on their rank of uncertainty value for the final prediction to
ensure the majority accurate principle in ensemble robustness and accuracy.
Compared with the previous dynamic method and staic adversarial traning model,
the presented approach can achieve significant robustness results without
damaging accuracy by combining dynamics and diversity property.