Cooperation or Competition: Avoiding Player Domination for Multi-Target
  Robustness via Adaptive Budgets
Yimu Wang Dinghuai Zhang Yihan Wu Heng Huang Hongyang Zhang
Despite incredible advances, deep learning has been shown to be susceptible
to adversarial attacks. Numerous approaches have been proposed to train robust
networks both empirically and certifiably. However, most of them defend against
only a single type of attack, while recent work takes steps forward in
defending against multiple attacks. In this paper, to understand multi-target
robustness, we view this problem as a bargaining game in which different
players (adversaries) negotiate to reach an agreement on a joint direction of
parameter updating. We identify a phenomenon named player domination in the
bargaining game, namely that the existing max-based approaches, such as MAX and
MSD, do not converge. Based on our theoretical analysis, we design a novel
framework that adjusts the budgets of different adversaries to avoid any player
dominance. Experiments on standard benchmarks show that employing the proposed
framework to the existing approaches significantly advances multi-target
robustness.