Pruning for Better Domain Generalizability
Xinglong Sun
In this paper, we investigate whether we could use pruning as a reliable
method to boost the generalization ability of the model. We found that existing
pruning method like L2 can already offer small improvement on the target domain
performance. We further propose a novel pruning scoring method, called DSS,
designed not to maintain source accuracy as typical pruning work, but to
directly enhance the robustness of the model. We conduct empirical experiments
to validate our method and demonstrate that it can be even combined with
state-of-the-art generalization work like MIRO(Cha et al., 2022) to further
boost the performance. On MNIST to MNIST-M, we could improve the baseline
performance by over 5 points by introducing 60% channel sparsity into the
model. On DomainBed benchmark and state-of-the-art MIRO, we can further boost
its performance by 1 point only by introducing 10% sparsity into the model.
Code can be found at:
https://github.com/AlexSunNik/Pruning-for-Better-Domain-Generalizability