![\"\"](\"https:\/\/www.hwcooling.net\/wp-content\/uploads\/2022\/03\/Sch\u00e9ma-GPU-Nvidia-GH100.png\")
<\/a>Like Nvidia’s previous compute GPUs, the Hopper\/H100 will use HBM-type memory, with a 6144-bit bus for six packages of this die-stacked memory (this is unchanged from Ampere). The memory can be either HBM3 or HBM2e, the memory controllers seem to support both. Coupled with the memory subsystem is the L2 cache, which the chip has a total of 60 MB.<\/p>\n In addition to these components, the GPU also has new connectivity. It supports PCI Express 5.0 as a first, but also NVLink 4. This interconnect has a bandwidth of 25 GB\/s like NVLink 3, but the effective signal frequency is said to be 100 Gbps per pin instead of 50 Gbps, while the number of parallel lanes per NVLink interface has dropped from four to two.<\/p>\n The GH100 GPU has 18 of these interfaces, while the Ampere has only 12. Therefore, the bandwidth that NVLink can transfer when using all of them has increased by 50 %, from 600 GB\/s for the Ampere GA100 to 900 GB\/s for the GH100.<\/p>\n The entire GPU is a monolithic die \u2013 it’s made as a single silicon. So the GPU is not chiplet-based, as sometimes reported in preliminary leaks<\/a>. The chip is 814 mm\u00b2, which is roughly the same size as the 12 nm Volta\/GV100 chip and smaller than the previous generation Ampere (GA100 is 826 mm\u00b2). It contains 80 billion transistors.<\/p>\n In the end, information that the chip could even be significantly larger than the so-called reticle limit<\/a> turned out to be false. But there is also another surprise \u2013 Nvidia is not producing this GPU on the 5nm process, but on its improved derivative, the 4nm process. It’s TSMC technology, but the process is said to be customized for Nvidia, who calls it “4N” (this copies the designation they used for Samsung’s modified 8nm process, so it’s a bit confusing \u2013 TSMC calls their process backwards, N4).<\/p>\n Updated:<\/strong> The 4N manufacturing process is, according to leaker Kopite7kimi<\/a>, possibly derived not from N4, but from the nominally 5nm N5P process. So Nvidia has renumbered its modified derivative a bit, but it’s probably not overly important, because both N4 and N5P are evolved variants of the same 5nm N5 process, just bent for different purposes. This may be why the information originally circulated that Hopper was a 5nm design. Yet the data stolen recently by hackers also seems to contain information about 5nm process node being used. It’s possible that the relabelling as “4N” may have come about after this hack, perhaps even in response to it.<\/em><\/p>\n Commercially sold models will have this GPU partially cut-down, as a few units need to be left off for redundancy. With a chip the size of the GH100, there will be relatively few silicon dies manufactured without any sort of manufacturing defect, so like consoles, it is expected from the start that part of the chip will be disabled in shipping products.<\/p>\n A more powerful of the variants the H100 accelerator will come in will be based on the proprietary SXM5<\/b> mezzanine format, which requires a special board and server. This version will have 132 active SM blocks (eight GPCs and 66 of the 72 TPCs), giving 16,896 shaders and 528 tensor cores. The frequency is expected to be somewhere around 1.78 GHz, but has not yet been finalized.<\/p>\n The GPU will use 80 GB of HBM3 memory, so again only five of the six stacks will be active, allowing Nvidia to reclaim even those manufactured GPUs where one of the HBM3 stacks has an error after completion. The memory bus is therefore only 5120 bits wide, which is also why the L2 cache is trimmed to 50MB.<\/p>\n The performance of this GPU is said to be up to 60 TFLOPS in FP64 precision calculations, but that’s with software tricks (using Tensor Core), the base FP64 performance is 30 TFLOPS (and 60 TFLOPS in FP32 single precision calculations). Tensor Core performance is claimed to be 500 TFLOPS in FP32 matrix calculations, 1000 TFLOPS in FP16 calculations, and FP8 format numbers are also supported, with performance up to 2000 TFLOPS \u2013 the same performance in the perhaps more practical INT8 integer format. This is just the pure physical computational performance of matrix operations, Nvidia quotes double that when using Structured Sparsity. Memory bandwidth is supposed to be up to 3 TB\/s when using HBM3.<\/p>\n But this accelerator will also have a very high power draw. TDP has already reached 400 W in the last generation, and the Nvidia H100 continues the inflation \u2013 the TDP of the SXM5 model is 700 W! Cooling servers, where there will be maybe eight of these modules side by side, will be no simple task.<\/p>\n There will also be a classic form-factor version in the form of a PCI Express 5.0 \u00d716 slot card. However, this model will have lower performance. The GPU will be trimmed down to just 14,592 shaders (114 SM) and 456 tensor cores. The memory will be kept at 80 GB, but older (and slower) HBM2e is to be used, so the total bandwidth will be only 2 TB\/s. The bus here is therefore also 5120-bit with 50MB L2 cache. NVLink connectivity is somewhat limited, the card will probably only have 12 interfaces (so only 600 GB\/s bandwidth). This card will have lower power draw, “only” 350W.<\/p>\n Nvidia lists performance to be at 80 % of the 700W SXM5 version \u2013 24 TFLOPS in FP64 calculations and 400 TFLOPS in FP32 matrix calculations on tensor cores and so on (800 TFLOPS in FP16, 1600 TFLOPS in FP8\/INT8).<\/p>\n It has already been mentioned that Hopper supports calculations in the FP8 data format, i.e. floating point numbers with only 8-bit of total precision, whereas until now 8-bit numbers were used only in integer format (which has better practical precision but small range). Support for FP8 is part of a new tensor core architecture that Nvidia refers to as the “Transformer Engine”. This supports the transition to this reduced precision, increasing performance.<\/p>\n When training AI models, this reduced accuracy can be used in place of FP16 where it is feasible (Nvidia’s software should do this automatically). This should be useful for so-called transformer neural networks. Moreover, the FP8 support can then also be used directly for inference on the Hopper GPU, with AI networks that were trained using it.<\/p>\n The Hopper architecture also brings support for the so-called dynamic programming<\/a> for the first time. This divides a task into smaller parts, whose partial results are then brought into the solution of the overall problem. This sub-division can allow for more optimal computation, as the processed subtask can be reused. The use of dynamic programming is enabled by the DPX instruction set extension, which is now making its debut in GPU Hopper.<\/p>\n Although the Hopper has already been unveiled (on Tuesday), this is not yet a hard launch. Nvidia always announces these GPUs “on paper” in advance. The H100 accelerators are expected to start selling with actual physical availability in the third quarter of this year. They will be available in servers from various manufacturers, but also in servers made directly by Nvidia, which bases the HGX series servers on them.<\/p>\n Sources: Nvidia<\/a>, AnandTech<\/a>, VideoCardz<\/a><\/i><\/p>\n English translation and edit by Jozef Dud\u00e1\u0161, original text by\u00a0Jan Ol\u0161an, editor for Cnews.cz<\/em><\/p>\n<\/a>PCIe 5.0 and NVLink 4<\/h3>\n
4 nm chip, 80 billion transistors<\/h3>\n
<\/a>700W SXM version and PCIe version<\/h3>\n
<\/a><\/a>H100 as a PCIe 5.0 card will be 350W<\/h3>\n
Architectural inovations: Transformer Engine, Dynamic Programming<\/h3>\n
<\/a>Availability in the second half of the year<\/h3>\n