Please note that this post is tagged as a rumor.
NVIDIA H100 with 64GB and 94GB memory
NVIDIA is reportedly adding more options based on the Hopper architecture.
According to the recent update to the PCI Device ID list, NVIDIA might add more variants of the H100 Hopper data-center accelerator. One of the contributors to the public PCI ID list claims that SXM5 based models are in the works featuring 94GB and 64GB of memory. The fact that these cards would utilize the SXM5 form factor would suggest we are looking at HBM memory, possibly of HBM3 design.
Nevertheless, the 94GB memory configuration is a bit unusual, and it is unclear how are HBM stacks configured exactly. However, this is not the first H100 card to launch with 94GB of memory, as there is a H100 NVL (for Large Language Models) with the same memory configuration. NVIDIA did not explain how 94GB memory is achieved, but it is likely due to HBM production and yields reasons. It may be more convenient to disable layers in the stack than the whole stack.
Additionally, a 64GB model is considered, which could be achieved with four out of six stacks featuring 16GB of capacity. What this essentially means that NVIDIA might soon have four memory options for its H100 GPU series and that’s probably not the full list.
It is worth keeping in mind that the H100 was also spotted with an even higher memory capacity of 120GB. Thus far, there was no official confirmation or even a driver PCI ID leak that would confirm such a variant though.
|VideoCardz.com NVIDIA Hopper Data-Center Series|
|PCI Device ID||SKU||Interface|
|2313||H100 CNX 80GB||PCIe Gen5 x16|
|2321||H100 NVL 94GB||SXM5|
|2322||H800 (China)||PCIe Gen5 x16|
|2331||H100||PCIe Gen5 x16|
|233A||H800L 94GB (China)||SXM5|
|????||H100 120GB||PCIe Gen5 x16|
Source: PCI ID list