Clearer Frames, Anytime: Resolving Velocity Ambiguity in Video Frame
  Interpolation
Zhihang Zhong Gurunandan Krishnan Xiao Sun Yu Qiao Sizhuo Ma Jian Wang
Existing video frame interpolation (VFI) methods blindly predict where each
object is at a specific timestep t ("time indexing"), which struggles to
predict precise object movements. Given two images of a baseball, there are
infinitely many possible trajectories: accelerating or decelerating, straight
or curved. This often results in blurry frames as the method averages out these
possibilities. Instead of forcing the network to learn this complicated
time-to-location mapping implicitly together with predicting the frames, we
provide the network with an explicit hint on how far the object has traveled
between start and end frames, a novel approach termed "distance indexing". This
method offers a clearer learning goal for models, reducing the uncertainty tied
to object speeds. We further observed that, even with this extra guidance,
objects can still be blurry especially when they are equally far from both
input frames (i.e., halfway in-between), due to the directional ambiguity in
long-range motion. To solve this, we propose an iterative reference-based
estimation strategy that breaks down a long-range prediction into several
short-range steps. When integrating our plug-and-play strategies into
state-of-the-art learning-based models, they exhibit markedly sharper outputs
and superior perceptual quality in arbitrary time interpolations, using a
uniform distance indexing map in the same format as time indexing.
Additionally, distance indexing can be specified pixel-wise, which enables
temporal manipulation of each object independently, offering a novel tool for
video editing tasks like re-timing.