NVIDIA Neural Texture Compression offers 4 times higher resolution than standard compression with 30% less memory

Published: May 5th 2023, 09:25 GMT   Comments

NVIDIA’s Neural Texture Compression for material texture compression

NVIDIA researchers have developed a novel compression algorithm for material textures.

In a paper titled “Random-Access Neural Compression of Material Textures”, NVIDIA presents a new algorithm for texture compression. The work targets the increasing requirements for computer memory, which now stores high-resolution textures as well as many properties and attributes attached to them to render high-fidelity and natural-looking materials.

The NTC is said to deliver 4 times higher resolution (16 more texels) than BC (Block Compression), which is a standard GPU-based texture compression available in many formats. NVIDIA’s algorithm represents textures as tensors (three dimensions), but without any assumptions like in block compression (such as channel count). The only thing that NTC assumes is that each texture has the same size.

Random and local access is an important feature of the NTC. For GPU texture compression, it is of utmost importance that textures can be accessed at a small cost without a delay, even when high compression rates are applied. This research focuses on compressing many channels and mipmaps (textures of different sizes) together. By doing so, the paper claims that the quality and bitrate are better than JPEG XL or AVIF formats.

Random-Access Neural Compression of Material Textures, NVIDIA

The continuous advancement of photorealism in rendering is accompanied by a growth in texture data and, consequently, increasing storage and memory demands. To address this issue, we propose a novel neural compression technique specifically designed for material textures. We unlock two more levels of detail, i.e., 16× more texels, using low bitrate compression, with image quality that is better than advanced image compression techniques, such as AVIF and JPEG XL. At the same time, our method allows for on-demand, real-time decompression with random access similar to block texture compression on GPUs. This extends our compression benefits all the way from disk storage to memory. The key idea behind our approach is compressing multiple material textures and their mipmap chains together, and using a small neural network, that is optimized for each material, to decompress them. Finally, we use a custom training implementation to achieve practical compression speeds, whose performance surpasses that of general frameworks, like PyTorch, by an order of magnitude.

— Random-Access Neural Compression of Material Textures, NVIDIA

Unlike common BCx algorithms, which require custom hardware, this algorithm utilizes the matrix multiplication methods, which are now accelerated by modern GPUs. According to the paper, this makes the NTC algorithm more practical and more capable due to lower disk and memory constraints.

Random-Access Neural Compression of Material Textures, NVIDIA

According to the paper, neural textures can be rendered in real-time with up to 16x more texel than BC approach. The cost of 4K render is 1.15ms which is higher than 0.49ms (measured on RTX 4090). More information will be presented at SIGGRAPH 2023 on August 6.

Source: NVIDIA

Comment Policy
  1. Comments must be written in English and should not exceed 1000 characters.
  2. Comments deemed to be spam or solely promotional in nature will be deleted. Including a link to relevant content is permitted, but comments should be relevant to the post topic. Discussions about politics are not allowed on this website.
  3. Comments and usernames containing language or concepts that could be deemed offensive will be deleted.
  4. Comments complaining about the post subject or its source will be removed.
  5. A failure to comply with these rules will result in a warning and, in extreme cases, a ban. In addition, please note that comments that attack or harass an individual directly will result in a ban without warning.
  6. VideoCardz has never been sponsored by AMD, Intel, or NVIDIA. Users claiming otherwise will be banned.
  7. VideoCardz Moderating Team reserves the right to edit or delete any comments submitted to the site without notice.
  8. If you have any questions about the commenting policy, please let us know through the Contact Page.
Hide Comment Policy