Random-Access Neural Compression of Material Textures
Karthik Vaidyanathan Marco Salvi Bartlomiej Wronski Tomas Akenine-Möller Pontus Ebelin Aaron Lefohn
The continuous advancement of photorealism in rendering is accompanied by a
growth in texture data and, consequently, increasing storage and memory
demands. To address this issue, we propose a novel neural compression technique
specifically designed for material textures. We unlock two more levels of
detail, i.e., 16x more texels, using low bitrate compression, with image
quality that is better than advanced image compression techniques, such as AVIF
and JPEG XL. At the same time, our method allows on-demand, real-time
decompression with random access similar to block texture compression on GPUs,
enabling compression on disk and memory. The key idea behind our approach is
compressing multiple material textures and their mipmap chains together, and
using a small neural network, that is optimized for each material, to
decompress them. Finally, we use a custom training implementation to achieve
practical compression speeds, whose performance surpasses that of general
frameworks, like PyTorch, by an order of magnitude.