Regenerating Arbitrary Video Sequences with Distillation Path-Finding
Thi-Ngoc-Hanh Le Sheng-Yi Yao Chun-Te Wu Tong-Yee Lee
If the video has long been mentioned as a widespread visualization form, the
animation sequence in the video is mentioned as storytelling for people.
Producing an animation requires intensive human labor from skilled professional
artists to obtain plausible animation in both content and motion direction,
incredibly for animations with complex content, multiple moving objects, and
dense movement. This paper presents an interactive framework to generate new
sequences according to the users' preference on the starting frame. The
critical contrast of our approach versus prior work and existing commercial
applications is that novel sequences with arbitrary starting frame are produced
by our system with a consistent degree in both content and motion direction. To
achieve this effectively, we first learn the feature correlation on the
frameset of the given video through a proposed network called RSFNet. Then, we
develop a novel path-finding algorithm, SDPF, which formulates the knowledge of
motion directions of the source video to estimate the smooth and plausible
sequences. The extensive experiments show that our framework can produce new
animations on the cartoon and natural scenes and advance prior works and
commercial applications to enable users to obtain more predictable results.