CATP: Cross-Attention Token Pruning for Accuracy Preserved Multimodal
  Model Inference
Ruqi Liao Chuqing Zhao Jin Li Weiqi Feng
In response to the rising interest in large multimodal models, we introduce
Cross-Attention Token Pruning (CATP), a precision-focused token pruning method.
Our approach leverages cross-attention layers in multimodal models, exemplified
by BLIP-2, to extract valuable information for token importance determination.
CATP employs a refined voting strategy across model heads and layers. In
evaluations, CATP achieves up to 12.1X higher accuracy compared to existing
token pruning methods, addressing the trade-off between computational
efficiency and model precision.