HSTR-Net: Reference Based Video Super-resolution for Aerial Surveillance
  with Dual Cameras
H. Umut Suluhan Hasan F. Ates Bahadir K. Gunturk
Aerial surveillance requires high spatio-temporal resolution (HSTR) video for
more accurate detection and tracking of objects. This is especially true for
wide-area surveillance (WAS), where the surveyed region is large and the
objects of interest are small. This paper proposes a dual camera system for the
generation of HSTR video using reference-based super-resolution (RefSR). One
camera captures high spatial resolution low frame rate (HSLF) video while the
other captures low spatial resolution high frame rate (LSHF) video
simultaneously for the same scene. A novel deep learning architecture is
proposed to fuse HSLF and LSHF video feeds and synthesize HSTR video frames at
the output. The proposed model combines optical flow estimation and
(channel-wise and spatial) attention mechanisms to capture the fine motion and
intricate dependencies between frames of the two video feeds. Simulations show
that the proposed model provides significant improvement over existing
reference-based SR techniques in terms of PSNR and SSIM metrics. The method
also exhibits sufficient frames per second (FPS) for WAS when deployed on a
power-constrained drone equipped with dual cameras.