NVIDIA H100 SXM with 4nm Hopper GPU and 80GB HBM3 memory has been pictured up close

NVIDIA H100 pictured up close

More than a month after GTC 2022 (conference), ServeTheHome is finally allowed to publish the photos of the NVIDIA’s  latest data-center GPU. 

NVIDIA H100 is an upcoming accelerator featuring 4 nm GPU made in Hopper architecture. Thus far we have only seen renders of this solution, but thanks to ServeTheHome we now get to see the new design of the SXM form factor in real photos. They confirm that the device has a board model called “PG520”:

NVIDIA H100 SXM 80GB GPU, Source: Patrick Kennedy/ServeTheHome

The H100 is using TSMC CoWoS packing technology with a 814 mm² large GH100 GPU die and six memory modules around. This variant features 16896 CUDA cores and ship with 80 GB of HBM3 memory. The SXM mezzanine connectors layout has changed compared to A100. Instead of two long connectors on each side of the GPU, one is now shorter.

This Hopper solution consumes up to 700W of power, which is 250W to 300W more than previous SXM data-center GPUs based on Ampere and Volta architectures.

NVIDIA H100 SXM 80GB GPU, Source: Patrick Kennedy/ServeTheHome

The H100 has recently appeared in Japan for pre-order at 33,000 USD. The only difference was that the card on sale was the PCIe Gen5 based model, whereas the card above is the SXM variant with more CUDA cores, more memory and higher power.

NVIDIA Data-Center GPUs Specifications
VideoCardz.com	NVIDIA H100	NVIDIA A100	NVIDIA Tesla V100	NVIDIA Tesla P100
Picture
GPU	GH100	GA100	GV100	GP100
Transistors	80B	54.2B	21.1B	15.3B
Die Size	814 mm²	828 mm²	815 mm²	610 mm²
Architecture	Hopper	Ampere	Volta	Pascal
Fabrication Node	TSMC N4	TSMC N7	12nm FFN	16nm FinFET+
GPU Clusters	132/114*	108	80	56
CUDA Cores	16896/14592*	6912	5120	3584
L2 Cache	50MB	40MB	6MB	4MB
Tensor Cores	528/456*	432	320	–
Memory Bus	5120-bit	5120-bit	4096-bit	4096-bit
Memory Size	80 GB HBM3/HBM2e*	40/80GB HBM2e	16/32 HBM2	16GB HBM2
TDP	700W/350W*	250W/300W/400W	250W/300W/450W	250W/300W
Interface	SXM5/*PCIe Gen5	SXM4/PCIe Gen4	SXM2/PCIe Gen3	SXM/PCIe Gen3
Launch Year	2022	2020	2017	2016

Source: ServeTheHome