Retro-FPN: Retrospective Feature Pyramid Network for Point Cloud
  Semantic Segmentation
Peng Xiang Xin Wen Yu-Shen Liu Hui Zhang Yi Fang Zhizhong Han
Learning per-point semantic features from the hierarchical feature pyramid is
essential for point cloud semantic segmentation. However, most previous methods
suffered from ambiguous region features or failed to refine per-point features
effectively, which leads to information loss and ambiguous semantic
identification. To resolve this, we propose Retro-FPN to model the per-point
feature prediction as an explicit and retrospective refining process, which
goes through all the pyramid layers to extract semantic features explicitly for
each point. Its key novelty is a retro-transformer for summarizing semantic
contexts from the previous layer and accordingly refining the features in the
current stage. In this way, the categorization of each point is conditioned on
its local semantic pattern. Specifically, the retro-transformer consists of a
local cross-attention block and a semantic gate unit. The cross-attention
serves to summarize the semantic pattern retrospectively from the previous
layer. And the gate unit carefully incorporates the summarized contexts and
refines the current semantic features. Retro-FPN is a pluggable neural network
that applies to hierarchical decoders. By integrating Retro-FPN with three
representative backbones, including both point-based and voxel-based methods,
we show that Retro-FPN can significantly improve performance over
state-of-the-art backbones. Comprehensive experiments on widely used benchmarks
can justify the effectiveness of our design. The source is available at
https://github.com/AllenXiangX/Retro-FPN