Task-Oriented Channel Attention for Fine-Grained Few-Shot Classification
SuBeen Lee WonJun Moon Hyun Seok Seong Jae-Pil Heo
The difficulty of the fine-grained image classification mainly comes from a
shared overall appearance across classes. Thus, recognizing discriminative
details, such as eyes and beaks for birds, is a key in the task. However, this
is particularly challenging when training data is limited. To address this, we
propose Task Discrepancy Maximization (TDM), a task-oriented channel attention
method tailored for fine-grained few-shot classification with two novel modules
Support Attention Module (SAM) and Query Attention Module (QAM). SAM highlights
channels encoding class-wise discriminative features, while QAM assigns higher
weights to object-relevant channels of the query. Based on these submodules,
TDM produces task-adaptive features by focusing on channels encoding
class-discriminative details and possessed by the query at the same time, for
accurate class-sensitive similarity measure between support and query
instances. While TDM influences high-level feature maps by task-adaptive
calibration of channel-wise importance, we further introduce Instance Attention
Module (IAM) operating in intermediate layers of feature extractors to
instance-wisely highlight object-relevant channels, by extending QAM. The
merits of TDM and IAM and their complementary benefits are experimentally
validated in fine-grained few-shot classification tasks. Moreover, IAM is also
shown to be effective in coarse-grained and cross-domain few-shot
classifications.