Meta-Tsallis-Entropy Minimization: A New Self-Training Approach for
  Domain Adaptation on Text Classification
Menglong Lu Zhen Huang Zhiliang Tian Yunxiang Zhao Xuanyu Fei Dongsheng Li
Text classification is a fundamental task for natural language processing,
and adapting text classification models across domains has broad applications.
Self-training generates pseudo-examples from the model's predictions and
iteratively trains on the pseudo-examples, i.e., minimizes the loss on the
source domain and the Gibbs entropy on the target domain. However, Gibbs
entropy is sensitive to prediction errors, and thus, self-training tends to
fail when the domain shift is large. In this paper, we propose Meta-Tsallis
Entropy minimization (MTEM), which applies a meta-learning algorithm to
optimize the instance adaptive Tsallis entropy on the target domain. To reduce
the computation cost of MTEM, we propose an approximation technique to
approximate the Second-order derivation involved in the meta-learning. To
efficiently generate pseudo labels, we propose an annealing sampling mechanism
for exploring the model's prediction probability. Theoretically, we prove the
convergence of the meta-learning algorithm in MTEM and analyze the
effectiveness of MTEM in achieving domain adaptation. Experimentally, MTEM
improves the adaptation performance of BERT with an average of 4 percent on the
benchmark dataset.