Methodology: performance tests
After the entry test of the Sapphire RX 6700 XT Nitro+, the second part of the series is here, tests with active Smart Access Memory. Immediately after the launch of the card, we would write about the performance that attacks the RTX 3070. In the meantime, all GeForce RTX 30 cards started to support Resizable BAR and the situation is changing a bit. ReBAR can no longer be ignored in Nvidia’s GPUs, but we don’t have to take it too seriously.
Before we get to compare the RX 6700 Nitro+ with active and inactive SAM, let’s make a small turn to GeForce. After the BIOS update on March 30, Resizable BAR with driver version 465.89 is now supported on all gaming graphics cards with Nvidia Ampere. It might seem like the time has come to discard all the old results without ReBAR, but that would be a mistake for several reasons.
With and without ReBAR
During the finalization of the methodology for testing graphics cards, which was not so long ago, we were discussing how to approach Smart Access Memory. We also considered a faster way to test only with SAM, as the cards already support it and it was their technological advantage. Of course, it depends on the platform or motherboard and processor you own.
In the end, I’m glad we decided to test both options – with active as well as inactive SAM. Also because this comparison showed two basic things, for which not only will we not discard the GeForce results without SAM out of the database, but we will even continue to execute them.
The first thing is that SAM optimizations are noticeable with each test, even to the level where newer results do not make sense when compared with older ones. We have already wondered about this in the article SAM tests on the RX 6800. AMD tunes more than it reveals. The smaller percentage improvement with active Resizable BAR than what Nvidia reports now was also shown by the time when only the RTX 3060 supported it. One of the options is to retest the cards, but this is impossible both because of the crazy size of tests, and because we do not dispose of the cards for a long time and we have to return them relatively quickly. But if the performance hasn’t changed that much over time, ReBAR degrades performance in some applications. Especially in those for computing, so it is good to keep track of which situations you’d be better off to turn it off.
It follows from the above that we will perform tests with active and inactive SAM or ReBAR (as you wish) on GeForce graphics as well. Just that we will probably not divide it into two articles anymore, but we’ll put everything into one. It will cost a huge amount of effort, but this comes to us as the right, uncompromising solution we like in the tests.
For the sake of consistency and not to be confusing, we will use the term Resizable BAR in the future, which is more suitable for GeForce (and it doesn’t matter with Radon), as SAM is more or less AMD’s marketing designation for the same thing. We will use the fact that we have interactive graphs for better orientation in the results. We’ve added “Resizable BAR” with on and off values among their sorting criteria. The default order is set according to them as well, the results will be grouped at the top of the graphs with ReBAR on, and ReBAR off will be below. Descending and ascending sorting by values is otherwise preserved.
If the presentation of the results does not seem clear enough to you, let us know and we will find a better solution. You can also arrange the bars in the graphs as you like. In the lower left corner of all graphs, there is a button via which you can set the sorting also according to GPU, test date or any other of the criteria.
Gaming tests
The largest sample of tests is from games. This is quite natural given that GeForce and Radeons, i.e. cards primarily intended for gaming use, will mostly be tested.
We chose the test games primarily to ensure the balance between the titles better optimized for the GPU of one manufacturer (AMD) or the other one (Nvidia). But we also took into account the popularity of the titles so that you could find your own results in the charts. Emphasis was also placed on genre diversity. Games such as RTS, FPS, TPS, car racing as well as a flight simulator, traditional RPG and sports games are represented by the most played football game. You can find a list of test games in the library of chapters (9–32), with each game having its own chapter, sometimes even two (chapters) for the best possible clarity, but this has its good reason, which we will share with you in the following text.
Before we start the gaming tests, each graphics card will pass the tests in 3D Mark to warm up to operating temperature. That’s good synthetics to start with.
We’re testing performance in games across three resolutions with an aspect ratio of 16:9 – FHD (1920 × 1080 px), QHD (2560 × 1440 px) and UHD (3840 × 2160 px) and always with the highest graphic settings, which can be set the same on all current GeForce and Radeon graphics cards. We turned off proprietary settings for the objectivity of the conclusions, and the settings with ray-tracing graphics are tested separately, as lower class GPUs do not support them. You will find their results in the complementary chapters. In addition to native ray-tracing, also after deploying Nvidia DLSS (2.0) and AMD FidelityFX CAS.
If the game has a built-in benchmark, we use that one (the only exception is Forza Horizon 4, where due to its instability – it used to crash here and there – we drive on our track), in other cases the measurements take place on the games’ own scenes. From those we capture the times of consecutive frames in tables (CSV) via OCAT, which FLAT interprets into intelligible fps speech. Both of these applications are from the workshop of colleagues from the gpureport.cz magazine. In addition to the average frame rate, we also write the minimum in the graphs. That contributes significantly to the overall gaming experience. For the highest possible accuracy, all measurements are repeated three times and the final results form their average value.
Computational tests
Testing the graphics card comprehensively, even in terms of computing power, is more difficult than drawing conclusions from the gaming environment. Just because such tests are usually associated with expensive software that you don’t just buy for the editorial office. On the other hand, we’ve found ways to bring the available computing performance to you. On the one hand, thanks to well-built benchmarks, on the other hand, there are also some freely available and at the same time relevant applications, and thirdly, we have invested something in the paid ones.
The tests begin with ComputeBench, which computes various simulations (including game graphics). Then we move on to the popular SPECviewperf benchmark (2020), which integrates partial operations from popular 2D and 3D applications, including 3Ds max and SolidWorks. Details on this test package can be found at spec.org. From the same team also comes SPECworkstation 3, where GPU acceleration is in the Caffe and Folding@Home tests. You can also find the results of the LuxMark 3.1 3D render in the graphs, and the remarkable GPGPU theoretical test also includes AIDA64 with FLOPS, IOPS and memory speed measurements.
For obvious reasons, 3D rendering makes the largest portion of the tests. This is also the case, for example, in the Blender practical tests (2.91). In addition to Cycles, we will also test the cards in Eevee and radeon ProRender renderers (let AMD have a related test, as most are optimized for Nvidia cards with proprietary CUDA and OptiX frameworks). Of course, an add-on for V-ray would also be interesting, but at the moment the editorial office can’t afford it, we may manage to get a “press” license in time, though, we’ll see. We want to expand application tests in the future. Definitely with some advanced AI testing (we haven’t come up with a reasonable way yet), including noise reduction (there would be some ideas already, but we haven’t incorporated those due to time constraints).
Graphics cards can also be tested well in photo editing. To get an idea of the performance in the popular Photoshop, we’re using a script in PugetBench, which simulates real work with various filters. Among them are those that use GPU acceleration. A comprehensive benchmark suggesting the performance of raster and vector graphics is then also used in alternative Affinity Photo. In Lightroom, there are remarkable color corrections (Enhance Details) of raw uncompressed photos. We apply these in batches to a 1 GB archive. All of these tasks can be accelerated by both GeForce and Radeon.
From another perspective, there are decryption tests in Hashcat with a selection of AES, MD5, NTLMv2, SHA1, SHA2-256/512 and WPA-EAPOL-PBKDF2 ciphers. Finally, in the OBS and XSplit broadcast applications, we measure how much the game performance will be reduced while recording. It is no longer provided by shaders, but by coders (AMD VCE and Nvidia Nvenc). These tests show how much spare performance each card has for typical online streaming.
There are, of course, more hardware acceleration options, typically for video editing and conversion. However, this is purely in the hands of encoders, which are always the same within one generation of cards from one manufacturer, so there is no point in testing them on every graphics card. It is different across generations and tests of this type will sooner or later appear. Just fine-tuning the metric is left, where the output will always have the same bitrate and pixel match. This is important for objective comparisons, because the encoder of one company/card may be faster in a particular profile with the same settings, but at the expense of the lower quality that another encoder has (but may not have, it’s just an example).
- Contents
- Methodology: performance tests
- Methodology: how we measure power draw
- Methodology: noise and sound measurement
- Methodology: temperature tests
- Test rig
- 3DMark
- Age of Empires II: DE
- Assassin’s Creed: Valhalla
- Battlefield V
- Battlefield V with DXR
- Borderlands 3
- Control
- Control with DXR
- Counter-Strike: GO
- Cyberpunk 2077
- Cyberpunk 2077 with FidelityFX CAS
- Cyberpunk 2077 with DXR
- DOOM Eternal
- F1 2020
- FIFA 21
- Forza Horizon 4
- Mafia: DE
- Metro Exodus
- Metro Exodus with DXR
- Microsoft Flight Simulator
- Red Dead Redemption 2 (Vulkan)
- Red Dead Redemption 2 (Dx12)
- Shadow of the Tomb Raider
- Shadow of the Tomb Raider with DXR
- Total War Saga: Troy
- Wasteland 3
- Overall game performance and performance per euro
- CompuBench (OpenCL)
- SPECviewperf 2020 and SPECworkstation 3
- FLOPS, IOPS and memory speed tests
- 3D rendering 1/2 (LuxMark and Blender@Cycles)
- 3D rendering 2/2 (Blender@Radeon ProRender and Eevee)
- Photo editing (Adobe Photoshop, Lightroom and Affinity Photo)
- Broadcasting (OBS and Xsplit)
- Password cracking
- GPU clock speed
- GPU temperatures
- Net graphics power draw and performance per watt
- Analysis of 12 V subcircuit power supply (higher load)
- Analysis of 12 V subcircuit power supply (lower load)
- Analysis of 3.3 V subcircuit power supply
- Noise level
- Frequency response of sound
- Conclusion
