Category Feature Transformer for Semantic Segmentation
Quan Tang Chuanjian Liu Fagui Liu Yifan Liu Jun Jiang Bowen Zhang Kai Han Yunhe Wang
Aggregation of multi-stage features has been revealed to play a significant
role in semantic segmentation. Unlike previous methods employing point-wise
summation or concatenation for feature aggregation, this study proposes the
Category Feature Transformer (CFT) that explores the flow of category embedding
and transformation among multi-stage features through the prevalent multi-head
attention mechanism. CFT learns unified feature embeddings for individual
semantic categories from high-level features during each aggregation process
and dynamically broadcasts them to high-resolution features. Integrating the
proposed CFT into a typical feature pyramid structure exhibits superior
performance over a broad range of backbone networks. We conduct extensive
experiments on popular semantic segmentation benchmarks. Specifically, the
proposed CFT obtains a compelling 55.1% mIoU with greatly reduced model
parameters and computations on the challenging ADE20K dataset.