A Multimodal Approach for Advanced Pest Detection and Classification
Jinli Duan Haoyu Ding Sung Kim
This paper presents a novel multi modal deep learning framework for enhanced
agricultural pest detection, combining tiny-BERT's natural language processing
with R-CNN and ResNet-18's image processing. Addressing limitations of
traditional CNN-based visual methods, this approach integrates textual context
for more accurate pest identification. The R-CNN and ResNet-18 integration
tackles deep CNN issues like vanishing gradients, while tiny-BERT ensures
computational efficiency. Employing ensemble learning with linear regression
and random forest models, the framework demonstrates superior discriminate
ability, as shown in ROC and AUC analyses. This multi modal approach, blending
text and image data, significantly boosts pest detection in agriculture. The
study highlights the potential of multi modal deep learning in complex
real-world scenarios, suggesting future expansions in diversity of datasets,
advanced data augmentation, and cross-modal attention mechanisms to enhance
model performance.