Revisiting Logistic-softmax Likelihood in Bayesian Meta-Learning for
  Few-Shot Classification
Tianjun Ke Haoqun Cao Zenan Ling Feng Zhou
Meta-learning has demonstrated promising results in few-shot classification
(FSC) by learning to solve new problems using prior knowledge. Bayesian methods
are effective at characterizing uncertainty in FSC, which is crucial in
high-risk fields. In this context, the logistic-softmax likelihood is often
employed as an alternative to the softmax likelihood in multi-class Gaussian
process classification due to its conditional conjugacy property. However, the
theoretical property of logistic-softmax is not clear and previous research
indicated that the inherent uncertainty of logistic-softmax leads to suboptimal
performance. To mitigate these issues, we revisit and redesign the
logistic-softmax likelihood, which enables control of the \textit{a priori}
confidence level through a temperature parameter. Furthermore, we theoretically
and empirically show that softmax can be viewed as a special case of
logistic-softmax and logistic-softmax induces a larger family of data
distribution than softmax. Utilizing modified logistic-softmax, we integrate
the data augmentation technique into the deep kernel based Gaussian process
meta-learning framework, and derive an analytical mean-field approximation for
task-specific updates. Our approach yields well-calibrated uncertainty
estimates and achieves comparable or superior results on standard benchmark
datasets. Code is publicly available at
\url{https://github.com/keanson/revisit-logistic-softmax}.