Unlimited Knowledge Distillation for Action Recognition in the Dark
Ruibing Jin Guosheng Lin Min Wu Jie Lin Zhengguo Li Xiaoli Li Zhenghua Chen
Dark videos often lose essential information, which causes the knowledge
learned by networks is not enough to accurately recognize actions. Existing
knowledge assembling methods require massive GPU memory to distill the
knowledge from multiple teacher models into a student model. In action
recognition, this drawback becomes serious due to much computation required by
video process. Constrained by limited computation source, these approaches are
infeasible. To address this issue, we propose an unlimited knowledge
distillation (UKD) in this paper. Compared with existing knowledge assembling
methods, our UKD can effectively assemble different knowledge without
introducing high GPU memory consumption. Thus, the number of teaching models
for distillation is unlimited. With our UKD, the network's learned knowledge
can be remarkably enriched. Our experiments show that the single stream network
distilled with our UKD even surpasses a two-stream network. Extensive
experiments are conducted on the ARID dataset.