NVIDIA preparing more H100 Hopper cards with 94GB and 64GB memory options

NVIDIA H100 with 64GB and 94GB memory

NVIDIA is reportedly adding more options based on the Hopper architecture.

According to the recent update to the PCI Device ID list, NVIDIA might add more variants of the H100 Hopper data-center accelerator. One of the contributors to the public PCI ID list claims that SXM5 based models are in the works featuring 94GB and 64GB of memory. The fact that these cards would utilize the SXM5 form factor would suggest we are looking at HBM memory, possibly of HBM3 design.

Nevertheless, the 94GB memory configuration is a bit unusual, and it is unclear how are HBM stacks configured exactly. However, this is not the first H100 card to launch with 94GB of memory, as there is a H100 NVL (for Large Language Models) with the same memory configuration. NVIDIA did not explain how 94GB memory is achieved, but it is likely due to HBM production and yields reasons. It may be more convenient to disable layers in the stack than the whole stack.

NVIDIA H100 94/64GB options, Source: PCI-ID list

Additionally, a 64GB model is considered, which could be achieved with four out of six stacks featuring 16GB of capacity. What this essentially means that NVIDIA might soon have four memory options for its H100 GPU series and that’s probably not the full list.

It is worth keeping in mind that the H100 was also spotted with an even higher memory capacity of 120GB. Thus far, there was no official confirmation or even a driver PCI ID leak that would confirm such a variant though.

Source: PCI ID list