Key Frame Extraction with Attention Based Deep Neural Networks
Samed Arslan Senem Tanberk
Automatic keyframe detection from videos is an exercise in selecting scenes
that can best summarize the content for long videos. Providing a summary of the
video is an important task to facilitate quick browsing and content
summarization. The resulting photos are used for automated works (e.g.
summarizing security footage, detecting different scenes used in music clips)
in different industries. In addition, processing high-volume videos in advanced
machine learning methods also creates resource costs. Keyframes obtained; It
can be used as an input feature to the methods and models to be used. In this
study; We propose a deep learning-based approach for keyframe detection using a
deep auto-encoder model with an attention layer. The proposed method first
extracts the features from the video frames using the encoder part of the
autoencoder and applies segmentation using the k-means clustering algorithm to
group these features and similar frames together. Then, keyframes are selected
from each cluster by selecting the frames closest to the center of the
clusters. The method was evaluated on the TVSUM video dataset and achieved a
classification accuracy of 0.77, indicating a higher success rate than many
existing methods. The proposed method offers a promising solution for key frame
extraction in video analysis and can be applied to various applications such as
video summarization and video retrieval.