Exploring Active Learning in Meta-Learning: Enhancing Context Set
  Labeling
Wonho Bae Jing Wang Danica J. Sutherland
Most meta-learning methods assume that the (very small) context set used to
establish a new task at test time is passively provided. In some settings,
however, it is feasible to actively select which points to label; the potential
gain from a careful choice is substantial, but the setting requires major
differences from typical active learning setups. We clarify the ways in which
active meta-learning can be used to label a context set, depending on which
parts of the meta-learning process use active learning. Within this framework,
we propose a natural algorithm based on fitting Gaussian mixtures for selecting
which points to label; though simple, the algorithm also has theoretical
motivation. The proposed algorithm outperforms state-of-the-art active learning
methods when used with various meta-learning algorithms across several
benchmark datasets.