Time-series Initialization and Conditioning for Video-agnostic
  Stabilization of Video Super-Resolution using Recurrent Networks
Hiroshi Mori Norimichi Ukita
A Recurrent Neural Network (RNN) for Video Super Resolution (VSR) is
generally trained with randomly clipped and cropped short videos extracted from
original training videos due to various challenges in learning RNNs. However,
since this RNN is optimized to super-resolve short videos, VSR of long videos
is degraded due to the domain gap. Our preliminary experiments reveal that such
degradation changes depending on the video properties, such as the video length
and dynamics. To avoid this degradation, this paper proposes the training
strategy of RNN for VSR that can work efficiently and stably independently of
the video length and dynamics. The proposed training strategy stabilizes VSR by
training a VSR network with various RNN hidden states changed depending on the
video properties. Since computing such a variety of hidden states is
time-consuming, this computational cost is reduced by reusing the hidden states
for efficient training. In addition, training stability is further improved
with frame-number conditioning. Our experimental results demonstrate that the
proposed method performed better than base methods in videos with various
lengths and dynamics.