Unified Entropy Optimization for Open-Set Test-Time Adaptation
Zhengqing Gao Xu-Yao Zhang Cheng-Lin Liu
Test-time adaptation (TTA) aims at adapting a model pre-trained on the
labeled source domain to the unlabeled target domain. Existing methods usually
focus on improving TTA performance under covariate shifts, while neglecting
semantic shifts. In this paper, we delve into a realistic open-set TTA setting
where the target domain may contain samples from unknown classes. Many
state-of-the-art closed-set TTA methods perform poorly when applied to open-set
scenarios, which can be attributed to the inaccurate estimation of data
distribution and model confidence. To address these issues, we propose a simple
but effective framework called unified entropy optimization (UniEnt), which is
capable of simultaneously adapting to covariate-shifted in-distribution (csID)
data and detecting covariate-shifted out-of-distribution (csOOD) data.
Specifically, UniEnt first mines pseudo-csID and pseudo-csOOD samples from test
data, followed by entropy minimization on the pseudo-csID data and entropy
maximization on the pseudo-csOOD data. Furthermore, we introduce UniEnt+ to
alleviate the noise caused by hard data partition leveraging sample-level
confidence. Extensive experiments on CIFAR benchmarks and Tiny-ImageNet-C show
the superiority of our framework. The code is available at
https://github.com/gaozhengqing/UniEnt