EfficientViT-SAM: Accelerated Segment Anything Model Without Performance
  Loss
Zhuoyang Zhang Han Cai Song Han
We present EfficientViT-SAM, a new family of accelerated segment anything
models. We retain SAM's lightweight prompt encoder and mask decoder while
replacing the heavy image encoder with EfficientViT. For the training, we begin
with the knowledge distillation from the SAM-ViT-H image encoder to
EfficientViT. Subsequently, we conduct end-to-end training on the SA-1B
dataset. Benefiting from EfficientViT's efficiency and capacity,
EfficientViT-SAM delivers 48.9x measured TensorRT speedup on A100 GPU over
SAM-ViT-H without sacrificing performance. Our code and pre-trained models are
released at https://github.com/mit-han-lab/efficientvit.