Rectifying Noisy Labels with Sequential Prior: Multi-Scale Temporal
  Feature Affinity Learning for Robust Video Segmentation
Beilei Cui Minqing Zhang Mengya Xu An Wang Wu Yuan Hongliang Ren
Noisy label problems are inevitably in existence within medical image
segmentation causing severe performance degradation. Previous segmentation
methods for noisy label problems only utilize a single image while the
potential of leveraging the correlation between images has been overlooked.
Especially for video segmentation, adjacent frames contain rich contextual
information beneficial in cognizing noisy labels. Based on two insights, we
propose a Multi-Scale Temporal Feature Affinity Learning (MS-TFAL) framework to
resolve noisy-labeled medical video segmentation issues. First, we argue the
sequential prior of videos is an effective reference, i.e., pixel-level
features from adjacent frames are close in distance for the same class and far
in distance otherwise. Therefore, Temporal Feature Affinity Learning (TFAL) is
devised to indicate possible noisy labels by evaluating the affinity between
pixels in two adjacent frames. We also notice that the noise distribution
exhibits considerable variations across video, image, and pixel levels. In this
way, we introduce Multi-Scale Supervision (MSS) to supervise the network from
three different perspectives by re-weighting and refining the samples. This
design enables the network to concentrate on clean samples in a coarse-to-fine
manner. Experiments with both synthetic and real-world label noise demonstrate
that our method outperforms recent state-of-the-art robust segmentation
approaches. Code is available at https://github.com/BeileiCui/MS-TFAL.