WeakSAM: Segment Anything Meets Weakly-supervised Instance-level
  Recognition
Lianghui Zhu Junwei Zhou Yan Liu Xin Hao Wenyu Liu Xinggang Wang
Weakly supervised visual recognition using inexact supervision is a critical
yet challenging learning problem. It significantly reduces human labeling costs
and traditionally relies on multi-instance learning and pseudo-labeling. This
paper introduces WeakSAM and solves the weakly-supervised object detection
(WSOD) and segmentation by utilizing the pre-learned world knowledge contained
in a vision foundation model, i.e., the Segment Anything Model (SAM). WeakSAM
addresses two critical limitations in traditional WSOD retraining, i.e., pseudo
ground truth (PGT) incompleteness and noisy PGT instances, through adaptive PGT
generation and Region of Interest (RoI) drop regularization. It also addresses
the SAM's problems of requiring prompts and category unawareness for automatic
object detection and segmentation. Our results indicate that WeakSAM
significantly surpasses previous state-of-the-art methods in WSOD and WSIS
benchmarks with large margins, i.e. average improvements of 7.4% and 8.5%,
respectively. The code is available at \url{https://github.com/hustvl/WeakSAM}.