Learning Kernel-Modulated Neural Representation for Efficient Light
  Field Compression
Jinglei Shi Yihong Xu Christine Guillemot
Light field is a type of image data that captures the 3D scene information by
recording light rays emitted from a scene at various orientations. It offers a
more immersive perception than classic 2D images but at the cost of huge data
volume. In this paper, we draw inspiration from the visual characteristics of
Sub-Aperture Images (SAIs) of light field and design a compact neural network
representation for the light field compression task. The network backbone takes
randomly initialized noise as input and is supervised on the SAIs of the target
light field. It is composed of two types of complementary kernels: descriptive
kernels (descriptors) that store scene description information learned during
training, and modulatory kernels (modulators) that control the rendering of
different SAIs from the queried perspectives. To further enhance compactness of
the network meanwhile retain high quality of the decoded light field, we
accordingly introduce modulator allocation and kernel tensor decomposition
mechanisms, followed by non-uniform quantization and lossless entropy coding
techniques, to finally form an efficient compression pipeline. Extensive
experiments demonstrate that our method outperforms other state-of-the-art
(SOTA) methods by a significant margin in the light field compression task.
Moreover, after aligning descriptors, the modulators learned from one light
field can be transferred to new light fields for rendering dense views,
indicating a potential solution for view synthesis task.