It’s roughly two weeks until AMD releases processors with the new Zen 5 architecture. This week, we finally got proper details on these CPUs’ actual architecture, which AMD revealed at the Tech Day event. So, we can now break down the changes the company has made to the core, compared to Zen 4 – and they’re pretty extensive, probably more so than they seemed in June. And AMD also reiterated its promise of a 16% increase in IPC for these CPUs.
Frontend: The biggest new feature of the core
What was known in advance about Zen 5 was that it would bring a significant boost to SIMD units which are getting their with doubled to 512-bit and also that there’s going to be a widening of the execution backend of the core. In the end, however, there are equally big changes to the frontend, which ensures that the execution units themselves have work to do, so in the core is significantly upgraded virtually in every part.
The most significant change in the frontend is that the core has duplicated blocks performing fetch and instruction decoding, not in the sense of simply doubling the bandwidth of the block, but that the core has these parts present in two parallel instances. The L1 instruction cache supports fetching program instructions from two locations (two instruction streams) simultaneously (the fetch handles 2× 32 bytes per cycle), to support this. Similarly, it is also possible to simultaneously fetch instructions from two locations from the uOP cache, which stores already decoded instructions.
These two instruction streams are consumed by two clusters of decoders. While all four previous generations of Zen cores have four decoders (i.e., the ability to decode four instructions per cycle), Zen 5 has two clusters each having four of these decoders, each capable of processing one of the two instruction streams.
This configuration can behave in one of two ways. The Zen 5 core still provides SMT unlike Intel’s latest architecture, so it can process two threads at once. In this situation, each thread gets one decoder, and thus has as many decoding resources as the previous cores had in total. However, AMD’s Mike Clark confirmed that both decoder clusters can be used by the core even when only a single thread is running (he also said that Zen cores are typically designed so that all the core’s resources are always available to a single running thread as well).
However, this capability likely has its limitations and the core is not able to involve both clusters as often as Intel’s E-Cores like Gracemont and Skymont, which also use similar technology. It is probably the case that a second cluster of decoders can get involved if there is a branch in the program and a program jump to another address is detected (predicted as taken branch). Thus, the second cluster of decoders does not have to wait for the decoders of the first cluster to finish their work, because it knows the place where it can start decoding, since the address is known to be start of an instruction.
It is sometimes claimed that there is one branch per six instructions in typical x86 code (on averaga), which would allow both clusters to be used relatively often, and so this addition may increase the overall IPC of the core by a bit. However, this probably has the effect that with the Zen 5 core, it is beneficial to compile code such that taken branches are the more frequently used patch during execution versus non-taken branches. This is something that may require profiling and recompiling of software to take advantage of, since in older architectures it is usually preferable if branches are mostly not taken, since earlier cores could handle fewer taken than non-taken branches.
However, the ability to use these two clusters of decoders simultaneously is likely to be improved in the future, so eventually the contribution to IPC is likely to grow. This may be one of those things where Zen 5 is building the infrastructure that will fully pay dividends in Zen 6 or 7 and so on.
The Zen 5 core has improved branch prediction to get better (lower) misprediction rate, which is something where improvements are made every generation. The prediction is not only supposed to have lower latencies, but also higher bandwidth – that is, the ability to process more branches per cycle.
2-ahead branch prediction
The branch predictor should actually be new, next-generation design based on the so-called 2-ahead predictor concept, and can handle conditional branches in zero bubble mode and can also handle two taken branches per cycle, which is a major upgrade. With the two taken branches, the prediction basically can go through three different branch prediction windows in code in one cycle. The L1 BTB (Branch Target Buffer) has been enlarged from 1.5K entries to massive 16K entries, TAGE is also larger. The core also received a larger 52entry return address stack, to improve performance during returns from function calls.
The decode phase is followed by the rename and dispatch phases, which have also been extended to handle eight instructions (operations) per cycle instead of six, so this is another part of the overall core widening. The very final retire phase handles 8 operations per cycle as well, so AMD doesn’t give it more capacity than the dispatch has, unlike Intel. However, the width of dispatch and rename (register renaming phase) is the same for AMD and Intel cores.
Smaller uOP cache might not actually have reduced capacity
Dispatch can be fed from the uOP cache in addition to the two decoder clusters. The uOP cache should have a capacity of 6000 entries, which is 50% more than Zen 2 and 3, but slightly less than Zen 4 (which apparently had 6750 entries). Up to 12 or 2×6 (already decoded) instructions per cycle can be served from this uOp cache.
Despite uOP cache shrinking, this may not be a regression. There’s an important change: whereas in Zen 4, one entry held one decoded uOP, in Zen 5 one entry holds the equivalent of an instruction, which can be fused uOPs. The uOPs are only split later for execution. This conserves the actual useful capacity of the uOP cache and also of dispatch bandwidth. It’s quite possibly this can make up for the drop in nominal entry capacity of uOP cache.
The associativity of the uOP cache was increased fro 12-way to 16-way too.
Still conservative depth of the Reorder Buffer
AMD has enlarged the Reorder Buffer (RoB), the main queue in which the Out-of-Order principle of instruction execution takes place and which forms an “instruction window” within which the processor can optimize execution by effectively reordering instructions. AMD has always had a relatively small RoB compared to Intel, let alone compared to Apple cores. Zen 3 had a RoB of 256 entries like Skylake (or Intel’s Gracemont E-Core, actually), Zen 4 increased it to 320 entries, while Intel’s Golden Cove already was at 512. Zen 5 will see a slightly bolder increase to 448 entries (+40%), but AMD is still more conservative here than all its competitors (Apple, Intel and ARM).
Zen 5 is now somewhere between the RoB depth of Sunny Cove/Ice Lake (352) and Intel’s Golden Cove cores, but it does look like AMD can develop cores with significantly more performance per 1 MHz with lower RoB depths than what Intel cores with simlar RoBs achieve. This may actually be fitting with the philosophy of “balance” between performance and efficiency that was touted as a guideline for Zen, back when the first generation was being announced in 2016 to 2017.
How many stages does the Zen 5 pipeline have?
AMD doesn’t actually share the pipeline (stage) depth of the Zen 5 core, which is something that wasn’t directly stated for most of the prior Zen cores either. So this will have to be derived by microbenchmarking analysis.
However, it seems that the pipeline may have been deepened by adding an extra stage somewhere. AMD confirmed that the misprediction penalty, which is usually and indirect guide to the number of pipeline stages in a processor, should on average be 1 cycle longer due to the changes made to the core.
The article continues on the next page.
⠀⠀
⠀
