NVIDIA teases next-gen B100 Blackwell GPU performance in GPT-3 175B Large Language Model

NVIDIA B100 Blackwell launching in 2024 with focus on AI

The company shares the first ‘graphs’ for its next-gen HPC accelerators. 

David Blackwell, Source: NVIDIA/College of Liberal Arts & Sciences

The B100 and GB200 GPUs are not standard graphics processors; they also go beyond the field of conventional graphics capabilities. These data-center accelerators now undertake complex tasks, specifically fueling the innovation around Large Language Models. NVIDIA anticipates a growing momentum and aims to secure its rightful market share in 2024.

The company confirms that the B100 GPU will be launched in the upcoming year. The latest slides from the SuperComputing 2023 Special Address confirm that Blackwell is indeed set to debut next year, despite not explicitly confirming this earlier. It should be noted that Blackwell HPC news should not transfer to a potential gaming version of this architecture. It exclusively refers to GB100, which is the initial generation of this architecture.

NVIDIA Data-center/AI GPU roadmap, Source: NVIDIA

NVIDIA indicates that B100 is set to deliver a substantial boost in inference for the GPT-3 model, featuring 178 billion parameters. This improvement is potentially double or more than what the existing H200 architecture already provides. However, it’s important to clarify that while the graph explicitly highlights Large Language Model (LLM) performance, this doesn’t necessarily translate to an equivalent level of raw compute power unless there’s a substantial increase in cores and power efficiency, an outcome that appears unlikely at that scale.

Moreover, NVIDIA affirms that B100 will witness an even higher increase in memory bandwidth. The current iteration of Hopper GPUs already utilizes HBM3e technology, offering up to 4.8 TB/s bandwidth. The forthcoming Blackwell chips are expected to continue employing this technology, possibly with additional stacks and increased bandwidth. NVIDIA emphasizes its commitment to designing architectures with future memory technology support in mind, as shown by integrating HBM3e with GH200 GPUs this year.

NVIDIA Data-center/AI GPU roadmap, Source: NVIDIA

NVIDIA has remained silent on any details regarding a potential gaming Blackwell architecture. Speculations, however, hint at the possibility of a new series of GB20x chips for the RTX 50 series, although the exact release timing remains undisclosed.

It’s worth noting that the nomenclature “Blackwell” is derived from David Harold Blackwell, an American statistician and mathematician recognized for his contributions to game theory, probability, and information theories. The successor to Blackwell architecture has not been named yet.

Source: NVIDIA