Exploring intra-task relations to improve meta-learning algorithms
Prabhat Agarwal Shreya Singh
Meta-learning has emerged as an effective methodology to model several
real-world tasks and problems due to its extraordinary effectiveness in the
low-data regime. There are many scenarios ranging from the classification of
rare diseases to language modelling of uncommon languages where the
availability of large datasets is rare. Similarly, for more broader scenarios
like self-driving, an autonomous vehicle needs to be trained to handle every
situation well. This requires training the ML model on a variety of tasks with
good quality data. But often times, we find that the data distribution across
various tasks is skewed, i.e.the data follows a long-tail distribution. This
leads to the model performing well on some tasks and not performing so well on
others leading to model robustness issues. Meta-learning has recently emerged
as a potential learning paradigm which can effectively learn from one task and
generalize that learning to unseen tasks. In this study, we aim to exploit
external knowledge of task relations to improve training stability via
effective mini-batching of tasks. We hypothesize that selecting a diverse set
of tasks in a mini-batch will lead to a better estimate of the full gradient
and hence will lead to a reduction of noise in training.