Nepali Video Captioning using CNN-RNN Architecture
Bipesh Subedi Saugat Singh Bal Krishna Bal
This article presents a study on Nepali video captioning using deep neural
networks. Through the integration of pre-trained CNNs and RNNs, the research
focuses on generating precise and contextually relevant captions for Nepali
videos. The approach involves dataset collection, data preprocessing, model
implementation, and evaluation. By enriching the MSVD dataset with Nepali
captions via Google Translate, the study trains various CNN-RNN architectures.
The research explores the effectiveness of CNNs (e.g., EfficientNetB0,
ResNet101, VGG16) paired with different RNN decoders like LSTM, GRU, and
BiLSTM. Evaluation involves BLEU and METEOR metrics, with the best model being
EfficientNetB0 + BiLSTM with 1024 hidden dimensions, achieving a BLEU-4 score
of 17 and METEOR score of 46. The article also outlines challenges and future
directions for advancing Nepali video captioning, offering a crucial resource
for further research in this area.