Improved Compositional Generalization by Generating Demonstrations for
  Meta-Learning
Sam Spilsbury Alexander Ilin
Meta-learning and few-shot prompting are viable methods to induce certain
types of compositional behaviour. However, these methods can be very sensitive
to the choice of support examples used. Choosing good supports from the
training data for a given test query is already a difficult problem, but in
some cases solving this may not even be enough. We consider a grounded language
learning problem (gSCAN) where good support examples for certain test splits
might not even exist in the training data, or would be infeasible to search
for. We design an agent which instead generates possible supports which are
relevant to the test query and current state of the world, then uses these
supports via meta-learning to solve the test query. We show substantially
improved performance on a previously unsolved compositional behaviour split
without a loss of performance on other splits. Further experiments show that in
this case, searching for relevant demonstrations even with an oracle function
is not sufficient to attain good performance when using meta-learning.