A successful application of convolutional architectures is to increase the resolution of single low-resolution images -- a image restoration task called super-resolution (SR). Naturally, SR is of value to resource constrained devices like mobile phones, electronic photograph frames and televisions to enhance image quality. However, SR demands perhaps the most extreme amounts of memory and compute operations of any mainstream vision task known today, preventing SR from being deployed to devices that require them. In this paper, we perform a early systematic study of system resource efficiency for SR, within the context of a variety of architectural and low-precision approaches originally developed for discriminative neural networks. We present a rich set of insights, representative SR architectures, and efficiency trade-offs; for example, the prioritization of ways to compress models to reach a specific memory and computation target and techniques to compact SR models so that they are suitable for DSPs and FPGAs. As a result of doing so, we manage to achieve better and comparable performance with previous models in the existing literature, highlighting the practicality of using existing efficiency techniques in SR tasks. Collectively, we believe these results provides the foundation for further research into the little explored area of resource efficiency for SR. Rapid progress has been made in the development of convolutional networks BID10 that are capable of taking a low-resolution image and producing an image with a significant increase in resolution. This image restoration task is referred to as super-resolution (SR) and has many potential applications in devices with limited memory and compute capacity. The fundamental problem however is that the state-of-the-art networks consist of thousands of layers and are some of the most resource intensive networks currently known. Furthermore, due to the spatial dimensions of feature maps needed to maintain or up-scale the input, the number of operations are counted in the billions as opposed to millions in models for discriminative tasks. As a result, there is a need for a general systematic approach to improve the efficiency of SR models.The challenge of the system resource requirements for deep learning models for tasks other than SR have been carefully studied in previous works BID62 BID40 BID48 , achieving massive gains in size and compute with little to no loss in performance. These reductions are achieved with a wide variety of methods being developed grounded in primarily architecture-level changes and techniques grounded in the use of low precision and quantized model parameters. However, how these efficiency methods behave when applied within SR have not yet been studied in significant depth, with very few results published in the literature. Extrapolating from prior results for other tasks is problematic given that predominantly existing studies are applied to discriminative tasks with substantially different architectures and operations. Due to the up-sampling structure of SR models, these efficiency methods may therefore produce potentially stronger side-effects to image distortion.In this paper, we detail a systematic study that seeks to bridge current understanding in SR and known approaches for scaling down the consumption of system resources by deep models. By examining the impact on image distortion quality when performing various efficiency techniques, we provide the following new insights:• The effectiveness of low rank tensor decomposition and other convolution approximations, which are comparable and successful in discriminative tasks, can vary considerably in SR.(See section 4.1).• Unlike image discriminative networks, SR networks suffer from a worse trade-off between efficiency and performance as more layers are compressed. (See section 4.2)• The practicality of adopting compression techniques for other tasks to SR as our best models are better or comparable to existing literature. For instance , our best model achieves significantly better performance and 6x less compute than MemNet BID51 and VDSR BID27 . Additionally , it also performs better and is 4.1x-5.8x smaller than SRMDNF BID61 . (See section 4.3)• Successful quantization techniques used in image discriminative tasks are equally successful in SR. (See section 5)
We build an understanding of resource-efficient techniques on Super-Resolution The paper proposes a detailed empirical evaluation of the trade-offs achieved by various convolutional neural networks on the super resolution problem. This paper proposed to improve the system resource efficiency for super resolution networks.