Active Learning for Video Classification with Frame Level Queries
Debanjan Goswami Shayok Chakraborty
Deep learning algorithms have pushed the boundaries of computer vision
research and have depicted commendable performance in a variety of
applications. However, training a robust deep neural network necessitates a
large amount of labeled training data, acquiring which involves significant
time and human effort. This problem is even more serious for an application
like video classification, where a human annotator has to watch an entire video
end-to-end to furnish a label. Active learning algorithms automatically
identify the most informative samples from large amounts of unlabeled data;
this tremendously reduces the human annotation effort in inducing a machine
learning model, as only the few samples that are identified by the algorithm,
need to be labeled manually. In this paper, we propose a novel active learning
framework for video classification, with the goal of further reducing the
labeling onus on the human annotators. Our framework identifies a batch of
exemplar videos, together with a set of informative frames for each video; the
human annotator needs to merely review the frames and provide a label for each
video. This involves much less manual work than watching the complete video to
come up with a label. We formulate a criterion based on uncertainty and
diversity to identify the informative videos and exploit representative
sampling techniques to extract a set of exemplar frames from each video. To the
best of our knowledge, this is the first research effort to develop an active
learning framework for video classification, where the annotators need to
inspect only a few frames to produce a label, rather than watching the
end-to-end video.