Segment Anything for comprehensive analysis of grapevine cluster
  architecture and berry properties
Efrain Torres-Lomas Jimena Lado-Jimena Guillermo Garcia-Zamora Luis Diaz-Garcia
Grape cluster architecture and compactness are complex traits influencing
disease susceptibility, fruit quality, and yield. Evaluation methods for these
traits include visual scoring, manual methodologies, and computer vision, with
the latter being the most scalable approach. Most of the existing computer
vision approaches for processing cluster images often rely on conventional
segmentation or machine learning with extensive training and limited
generalization. The Segment Anything Model (SAM), a novel foundation model
trained on a massive image dataset, enables automated object segmentation
without additional training. This study demonstrates out-of-the-box SAM's high
accuracy in identifying individual berries in 2D cluster images. Using this
model, we managed to segment approximately 3,500 cluster images, generating
over 150,000 berry masks, each linked with spatial coordinates within their
clusters. The correlation between human-identified berries and SAM predictions
was very strong (Pearson r2=0.96). Although the visible berry count in images
typically underestimates the actual cluster berry count due to visibility
issues, we demonstrated that this discrepancy could be adjusted using a linear
regression model (adjusted R2=0.87). We emphasized the critical importance of
the angle at which the cluster is imaged, noting its substantial effect on
berry counts and architecture. We proposed different approaches in which berry
location information facilitated the calculation of complex features related to
cluster architecture and compactness. Finally, we discussed SAM's potential
integration into currently available pipelines for image generation and
processing in vineyard conditions.