NVIDIA’s Neural Texture Compression for material texture compression
NVIDIA researchers have developed a novel compression algorithm for material textures.
In a paper titled “Random-Access Neural Compression of Material Textures”, NVIDIA presents a new algorithm for texture compression. The work targets the increasing requirements for computer memory, which now stores high-resolution textures as well as many properties and attributes attached to them to render high-fidelity and natural-looking materials.
The NTC is said to deliver 4 times higher resolution (16 more texels) than BC (Block Compression), which is a standard GPU-based texture compression available in many formats. NVIDIA’s algorithm represents textures as tensors (three dimensions), but without any assumptions like in block compression (such as channel count). The only thing that NTC assumes is that each texture has the same size.
Random and local access is an important feature of the NTC. For GPU texture compression, it is of utmost importance that textures can be accessed at a small cost without a delay, even when high compression rates are applied. This research focuses on compressing many channels and mipmaps (textures of different sizes) together. By doing so, the paper claims that the quality and bitrate are better than JPEG XL or AVIF formats.
Random-Access Neural Compression of Material Textures, NVIDIA
The continuous advancement of photorealism in rendering is accompanied by a growth in texture data and, consequently, increasing storage and memory demands. To address this issue, we propose a novel neural compression technique specifically designed for material textures. We unlock two more levels of detail, i.e., 16× more texels, using low bitrate compression, with image quality that is better than advanced image compression techniques, such as AVIF and JPEG XL. At the same time, our method allows for on-demand, real-time decompression with random access similar to block texture compression on GPUs. This extends our compression benefits all the way from disk storage to memory. The key idea behind our approach is compressing multiple material textures and their mipmap chains together, and using a small neural network, that is optimized for each material, to decompress them. Finally, we use a custom training implementation to achieve practical compression speeds, whose performance surpasses that of general frameworks, like PyTorch, by an order of magnitude.
— Random-Access Neural Compression of Material Textures, NVIDIA
Unlike common BCx algorithms, which require custom hardware, this algorithm utilizes the matrix multiplication methods, which are now accelerated by modern GPUs. According to the paper, this makes the NTC algorithm more practical and more capable due to lower disk and memory constraints.
Random-Access Neural Compression of Material Textures, NVIDIA
According to the paper, neural textures can be rendered in real-time with up to 16x more texel than BC approach. The cost of 4K render is 1.15ms which is higher than 0.49ms (measured on RTX 4090). More information will be presented at SIGGRAPH 2023 on August 6.
Source: NVIDIA
