A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigm
Qi Wang Yiqin Lv Yanghe Feng Zheng Xie Jincai Huang
Meta learning is a promising paradigm to enable skill transfer across tasks.
Most previous methods employ the empirical risk minimization principle in
optimization. However, the resulting worst fast adaptation to a subset of tasks
can be catastrophic in risk-sensitive scenarios. To robustify fast adaptation,
this paper optimizes meta learning pipelines from a distributionally robust
perspective and meta trains models with the measure of expected tail risk. We
take the two-stage strategy as heuristics to solve the robust meta learning
problem, controlling the worst fast adaptation cases at a certain probabilistic
level. Experimental results show that our simple method can improve the
robustness of meta learning to task distributions and reduce the conditional
expectation of the worst fast adaptation risk.