FuseSR: Super Resolution for Real-time Rendering through Efficient
  Multi-resolution Fusion
Zhihua Zhong Jingsen Zhu Yuxin Dai Chuankun Zheng Yuchi Huo Guanlin Chen Hujun Bao Rui Wang
The workload of real-time rendering is steeply increasing as the demand for
high resolution, high refresh rates, and high realism rises, overwhelming most
graphics cards. To mitigate this problem, one of the most popular solutions is
to render images at a low resolution to reduce rendering overhead, and then
manage to accurately upsample the low-resolution rendered image to the target
resolution, a.k.a. super-resolution techniques. Most existing methods focus on
exploiting information from low-resolution inputs, such as historical frames.
The absence of high frequency details in those LR inputs makes them hard to
recover fine details in their high-resolution predictions. In this paper, we
propose an efficient and effective super-resolution method that predicts
high-quality upsampled reconstructions utilizing low-cost high-resolution
auxiliary G-Buffers as additional input. With LR images and HR G-buffers as
input, the network requires to align and fuse features at multi resolution
levels. We introduce an efficient and effective H-Net architecture to solve
this problem and significantly reduce rendering overhead without noticeable
quality deterioration. Experiments show that our method is able to produce
temporally consistent reconstructions in $4 \times 4$ and even challenging $8
\times 8$ upsampling cases at 4K resolution with real-time performance, with
substantially improved quality and significant performance boost compared to
existing works.