Ophthalmic Biomarker Detection Using Ensembled Vision Transformers --
  Winning Solution to IEEE SPS VIP Cup 2023
H. A. Z. Sameen Shahgir Khondker Salman Sayeed Tanjeem Azwad Zaman Md. Asif Haider Sheikh Saifur Rahman Jony M. Sohel Rahman
This report outlines our approach in the IEEE SPS VIP Cup 2023: Ophthalmic
Biomarker Detection competition. Our primary objective in this competition was
to identify biomarkers from Optical Coherence Tomography (OCT) images obtained
from a diverse range of patients. Using robust augmentations and 5-fold
cross-validation, we trained two vision transformer-based models: MaxViT and
EVA-02, and ensembled them at inference time. We find MaxViT's use of
convolution layers followed by strided attention to be better suited for the
detection of local features while EVA-02's use of normal attention mechanism
and knowledge distillation is better for detecting global features. Ours was
the best-performing solution in the competition, achieving a patient-wise F1
score of 0.814 in the first phase and 0.8527 in the second and final phase of
VIP Cup 2023, scoring 3.8% higher than the next-best solution.