Domain Generalisation via Risk Distribution Matching
Toan Nguyen Kien Do Bao Duong Thin Nguyen
We propose a novel approach for domain generalisation (DG) leveraging risk
distributions to characterise domains, thereby achieving domain invariance. In
our findings, risk distributions effectively highlight differences between
training domains and reveal their inherent complexities. In testing, we may
observe similar, or potentially intensifying in magnitude, divergences between
risk distributions. Hence, we propose a compelling proposition: Minimising the
divergences between risk distributions across training domains leads to robust
invariance for DG. The key rationale behind this concept is that a model,
trained on domain-invariant or stable features, may consistently produce
similar risk distributions across various domains. Building upon this idea, we
propose Risk Distribution Matching (RDM). Using the maximum mean discrepancy
(MMD) distance, RDM aims to minimise the variance of risk distributions across
training domains. However, when the number of domains increases, the direct
optimisation of variance leads to linear growth in MMD computations, resulting
in inefficiency. Instead, we propose an approximation that requires only one
MMD computation, by aligning just two distributions: that of the worst-case
domain and the aggregated distribution from all domains. Notably, this method
empirically outperforms optimising distributional variance while being
computationally more efficient. Unlike conventional DG matching algorithms, RDM
stands out for its enhanced efficacy by concentrating on scalar risk
distributions, sidestepping the pitfalls of high-dimensional challenges seen in
feature or gradient matching. Our extensive experiments on standard benchmark
datasets demonstrate that RDM shows superior generalisation capability over
state-of-the-art DG methods.