Fourier Test-time Adaptation with Multi-level Consistency for Robust
  Classification
Yuhao Huang Xin Yang Xiaoqiong Huang Xinrui Zhou Haozhe Chi Haoran Dou Xindi Hu Jian Wang Xuedong Deng Dong Ni
Deep classifiers may encounter significant performance degradation when
processing unseen testing data from varying centers, vendors, and protocols.
Ensuring the robustness of deep models against these domain shifts is crucial
for their widespread clinical application. In this study, we propose a novel
approach called Fourier Test-time Adaptation (FTTA), which employs a
dual-adaptation design to integrate input and model tuning, thereby jointly
improving the model robustness. The main idea of FTTA is to build a reliable
multi-level consistency measurement of paired inputs for achieving
self-correction of prediction. Our contribution is two-fold. First, we
encourage consistency in global features and local attention maps between the
two transformed images of the same input. Here, the transformation refers to
Fourier-based input adaptation, which can transfer one unseen image into source
style to reduce the domain gap. Furthermore, we leverage style-interpolated
images to enhance the global and local features with learnable parameters,
which can smooth the consistency measurement and accelerate convergence.
Second, we introduce a regularization technique that utilizes style
interpolation consistency in the frequency space to encourage self-consistency
in the logit space of the model output. This regularization provides strong
self-supervised signals for robustness enhancement. FTTA was extensively
validated on three large classification datasets with different modalities and
organs. Experimental results show that FTTA is general and outperforms other
strong state-of-the-art methods.