HBM3: the highest-performance memory for GPUs, bandwidth up to 5 TB/s?
GDDR7 is still far in the future (though Nvidia got ahead with their own Micron-made GDDR6X). But high-performance GPUs could get another new memory technology in the near future: HBM3 chips. Hynix has now announced that it has developed the first generation of this memory, which could lift graphics bandwidth by almost an order of magnitude. A single HBM3 chip (package) has a bandwidth higher than the entire RTX 3080.
SK Hynix, the company behind the first ever generation of this technology (referred to as just “HBM” back then), has now announced the development of a third generation of the technology. Or rather, the fourth, if we consider HBM2E, which increased frequencies and capacities compared to HBM2 but otherwise did not differ that much, as a separate version.
HBM3 memory has been talked about prospectively in 2016, but only now it is slowly getting ready for the real world. In the summer, the availability of IP for memory controllers from Synopsys and Rambus was announced, and now the memory has been announced as well. However, it seems that the standard is not yet finally ratified, we will probably have to wait until sometime in 2022 for the arrival of finished devices (GPUs, CPUs and accelerators) with this memory.
Giant bandwidth
HBM3 will naturally increase performance. And by quite a bit. It’s expected to have an effective frequency of up to 6400 MHz, which is on par with LPDDR5, GDDR5, or overclocked tiers of the upcoming DDR5. HBM started out at relatively low frequencies, but this deficit is gradually easing. This clock speed would mean that the memory would have 3.2x better bandwidth than the 2.0GHz (effectively) HBM2 as used in the AMD Radeon VII card, for example. It’s also a two-fold speedup against the 3.2GHz-running HBM2E, although there have been faster (3.6GHz) variants beyond the standard since then.
HBM3 memories should again be produced in the form of multi-chip packages with a very wide bus (it has 1024 bits), so in this case one “chip” is not one silicon, but several dies stacked inside. One such package can replace several GDDR6 chips. For example, graphics cards could theoretically have only a single package.
Should the HBM3runs at the announced maximum clock speed of 6400 MHz effectively (or if you want, 6400 MT/s per pin/bit), then a single package will achieve a bandwidth of 819.2 GB/s, which is on the level of today’s high-end graphics card – Nvidia GeForce RTX 3080 has a lower bandwidth (760 GB/s), GeForce RTX 3080 Ti has more (912 GB/s). And that 819 GB/s, by the way, is double the bandwidth of the monstrous integrated GPU of the Apple M1 Max processor, which impressed the world with a 512-bit LPDDR5-6400 memory controller providing 400 GB/s of bandwidth (more accurately, it’s about 409 GB/s). Compared to the capabilities of HBM3, however, that doesn’t seem like that much, anymore.
If a GPU was equipped with just two of these packages (like the Radeon RX Vega 56/64 once was), it would have a bandwidth of 1.638 TB/s, far more than the most powerful gaming graphics to date. With four packages (4096-bit bus), we’re already at 3.276 TB/s, and if we even use six packages (6144-bit bus), which is what Nvidia’s high-end compute GPUs have been supporting lately, we get almost 5 TB/s.
With HBM3, higher capacities should also be available for GPUs or other processors. Hynix’s now-developed HBM3 memory will come in 16GB and 24GB packages that hold 12 or 8 layers of DRAM. Each 16Gb/2GB DRAM dies is only about 30 micrometers thick and they are connected by TSVs (vertical wires running through the chip).
Tip: Ampere GPUs have new PAM4-based GDDR6X memory, we have details
Maybe in Nvidia Hopper
The most likely users of HBM3 memory will be manufacturers of AI accelerators and similar hardware. This could include specialized ASICs, but of course also Nvidia’s highest-performance compute GPUs, perhaps even the upcoming Hopper GPU, supposedly chiplet-based and possibly very large, as they may consume over 1000 W. That would imply a huge amount of execution units and also compute power, so maybe such a monster might actually need such insane memory bandwidth, like those aforementioned six HBM3 packages with 5 TB/s.
Although who knows, maybe Nvidia will jump to even more packages, there probably aren’t any specific limits here, for example Intel’s upcoming Sapphire Rapids Xeons integrating HBM2E memory have up to eight packages under the hood (i.e. 8192-bit bus). Also, processors could theoretically have HBM3 integrated in their package too, so we might see it in some future generation of Xeons.
Will HBM3 ever be in gaming graphics?
Gaming graphics cards have stopped using HBM and HBM2 memory since the departure of the Radeon R9 Fury and then the Radeon RX Vega 56 and 64 cards (the only exception was the Radeon Pro 5600M with AMD’s Navi 12 chip used exclusively by Apple). The reason is that these memories have to be mounted on a silicon interposer, which makes them very expensive to use. That’s why they are now used only in server GPUs, which sell for significantly higher prices than gaming cards (…or at least they did before the cryptocurrency bubble). However, it’s probably not entirely out of the question that HBM-type memory technology could make a comeback into gaming graphics cards.
The advent of advanced packaging techniques such as Intel’s EMIB could also help. These are silicon bridges that only connect the GPU and HBM2 memory under the chips. There is no longer a need for a large area interposer, so the whole thing is less expensive. Other companies should have similar technologies, not just Intel, so it is possible that in the future the use of HBM3 will become much cheaper and there will be opportunity to deploy the technology on gaming graphics again.
The power savings that HBM should bring against relatively power-hungry GDDR-type memories (GDDR6X in particular seems to increase the TDP of GeForce RTX 3000 cards quite significantly) could then be turned into a boost for the performance of the GPU itself, as there would be more power budget remaining for it. Alternatively, HBM3 could enable better GPU performance in gaming laptops due to its size and again lower power consumption.
Jan Olšan, editor for Cnews.cz
English translation and edit by Jozef Dudáš, original text by Jan Olšan, editor for Cnews.cz
