NVIDIA announces world’s largest GPU cluster with 81920 CUDA cores

Published: 27th Mar 2018, 17:14 GMT   Comments

NVIDIA Volta GPUs connected by NVSwitch create world’s largest ‘GPU’

The ‘GPU’, which is actually a cluster with 81920 CUDA Cores in total and up to 2K TFLOPs with Tensor Core computing, is actually a cluster of 16 Tesla V100 32GB GPUs, which are connected by NVSwitch.

This system has 512 GB of HBM2 memory and total bandwidth up to 14.4 TB/s. The ‘GPU’ is offered in NVIDIA DGX2 system, which Jensen calls ‘world’s largest graphics card’.

NVSwitch: A Revolutionary Interconnect Fabric
NVSwitch offers 5x higher bandwidth than the best PCIe switch, allowing developers to build systems with more GPUs hyperconnected to each other. It will help developers break through previous system limitations and run much larger datasets. It also opens the door to larger, more complex workloads, including modeling parallel training of neural networks.

NVSwitch extends the innovations made available through NVIDIA NVLink™, the first high-speed interconnect technology developed by NVIDIA. NVSwitch allows system designers to build even more advanced systems that can flexibly connect any topology of NVLink-based GPUs.

NVIDIA DGX-2: World’s First Two Petaflop System 
NVIDIA’s new DGX-2 system reached the two petaflop milestone by drawing from a wide range of industry-leading technology advances developed by NVIDIA at all levels of the computing stack.

DGX-2 is the first system to debut NVSwitch, which enables all 16 GPUs in the system to share a unified memory space. Developers now have the deep learning training power to tackle the largest datasets and most complex deep learning models.

Combined with a fully optimized, updated suite of NVIDIA deep learning software, DGX-2 is purpose-built for data scientists pushing the outer limits of deep learning research and computing.

DGX-2 can train FAIRSeq, a state-of-the-art neural machine translation model, in less than two days — a 10x improvement in performance from the DGX-1 with Volta, introduced in September.




Comment Policy
  • Comments must be written in English.
  • Comments deemed to be spam or solely promotional in nature will be deleted. Including a link to relevant content is permitted, but comments should be relevant to the post topic.
  • Comments containing language or concepts that could be deemed offensive will be deleted. Note this may include abusive, threatening, pornographic, offensive, misleading or libelous language.
  • A failure to comply with these rules will result in a warning and, in extreme cases, a ban.
  • Please note that comments that attack or harass an individual directly will be deleted and such comments will result in a ban.
  • VideoCardz Moderating Team reserves the right to edit or delete any comments submitted to the site without notice.
  • If you have any questions about the commenting policy, please let us know through the Contact Page.
Hide Comment Policy
Comments