Generative Adversarial Shaders for Real-Time Realism Enhancement
Arturo Salmi Szabolcs Cséfalvay James Imber
Application of realism enhancement methods, particularly in real-time and
resource-constrained settings, has been frustrated by the expense of existing
methods. These achieve high quality results only at the cost of long runtimes
and high bandwidth, memory, and power requirements. We present an efficient
alternative: a high-performance, generative shader-based approach that adapts
machine learning techniques to real-time applications, even in
resource-constrained settings such as embedded and mobile GPUs. The proposed
learnable shader pipeline comprises differentiable functions that can be
trained in an end-to-end manner using an adversarial objective, allowing for
faithful reproduction of the appearance of a target image set without manual
tuning. The shader pipeline is optimized for highly efficient execution on the
target device, providing temporally stable, faster-than-real time results with
quality competitive with many neural network-based methods.