Task Aware Modulation using Representation Learning: An Approach for Few
  Shot Learning in Heterogeneous Systems
Arvind Renganathan Rahul Ghosh Ankush Khandelwal Vipin Kumar
We present a Task-aware modulation using Representation Learning (TAM-RL)
framework that enhances personalized predictions in few-shot settings for
heterogeneous systems when individual task characteristics are not known.
TAM-RL extracts embeddings representing the actual inherent characteristics of
these entities and uses these characteristics to personalize the predictions
for each entity/task. Using real-world hydrological and flux tower benchmark
data sets, we show that TAM-RL can significantly outperform existing baseline
approaches such as MAML and multi-modal MAML (MMAML) while being much faster
and simpler to train due to less complexity. Specifically, TAM-RL eliminates
the need for sensitive hyper-parameters like inner loop steps and inner loop
learning rate, which are crucial for model convergence in MAML, MMAML. We
further present an empirical evaluation via synthetic data to explore the
impact of heterogeneity amongst the entities on the relative performance of
MAML, MMAML, and TAM-RL. We show that TAM-RL significantly improves predictive
performance for cases where it is possible to learn distinct representations
for different tasks.