Methodology: performance tests
Without application/computational tests, graphics card tests would be incomplete. Therefore, we will focus on this area outside of gaming that most hardware magazines neglect. We understand the reasons, there are several of them, but even so, it is possible with a little effort to make at least a few measurements. So hopefully they will also help multimedia creators in choosing the right graphics card.
Methodology: performance tests
Testing the graphics card comprehensively, even in terms of computing power, is more difficult than drawing conclusions from the gaming environment. Just because such tests are usually associated with expensive software that you don’t just buy for the editorial office. On the other hand, we’ve found ways to bring the available computing performance to you. On the one hand, thanks to well-built benchmarks, on the other hand, there are also some freely available and at the same time relevant applications, and thirdly, we have invested something in the paid ones.
The tests begin with ComputeBench, which computes various simulations (including game graphics). Then we move on to the popular SPECviewperf benchmark (2020), which integrates partial operations from popular 2D and 3D applications, including 3Ds max and SolidWorks. Details on this test package can be found at spec.org. From the same team also comes SPECworkstation 3, where GPU acceleration is in the Caffe and Folding@Home tests. You can also find the results of the LuxMark 3.1 3D render in the graphs, and the remarkable GPGPU theoretical test also includes AIDA64 with FLOPS, IOPS and memory speed measurements.
For obvious reasons, 3D rendering makes the largest portion of the tests. This is also the case, for example, in the Blender practical tests (2.91). In addition to Cycles, we will also test the cards in Eevee and radeon ProRender renderers (let AMD have a related test, as most are optimized for Nvidia cards with proprietary CUDA and OptiX frameworks). Of course, an add-on for V-ray would also be interesting, but at the moment the editorial office can’t afford it, we may manage to get a “press” license in time, though, we’ll see. We want to expand application tests in the future. Definitely with some advanced AI testing (we haven’t come up with a reasonable way yet), including noise reduction (there would be some ideas already, but we haven’t incorporated those due to time constraints).
Graphics cards can also be tested well with photo editing. There is already a large number of various filters with GPU acceleration support, but the possibility of convenient repeated measurements is important. This quite well allows blur in Photoshop and in the cheaper Affinity. We implement it on a large photo with a resolution of 62 Mpx, to which we apply a script via Macro Recorder with a high frequency of steps there (250 px) and back (0 px), while recording average fps. In Lightroom, there are notable color corrections (Enhance Details) of raw uncompressed photos. We apply these in batches to a 1 GB archive. All of these tasks can be accelerated by both GeForce and Radeon.
From another perspective, there are decryption tests in Hashcat with a selection of AES, MD5, NTLMv2, SHA1, SHA2-256/512 and WPA-EAPOL-PBKDF2 ciphers. Finally, in the OBS and XSplit broadcast applications, we measure how much the game performance will be reduced while recording. It is no longer provided by shaders, but by coders (AMD VCE and Nvidia Nvenc). These tests show how much spare performance each card has for typical online streaming.
There are, of course, more hardware acceleration options, typically for video editing and conversion. However, this is purely in the hands of encoders, which are always the same within one generation of cards from one manufacturer, so there is no point in testing them on every graphics card. It is different across generations and tests of this type will sooner or later appear. Just fine-tuning the metric is left, where the output will always have the same bitrate and pixel match. This is important for objective comparisons, because the encoder of one company/card may be faster in a particular profile with the same settings, but at the expense of the lower quality that another encoder has (but may not have, it’s just an example).
- Contents
- Methodology: performance tests
- Methodology: how we measure power draw
- Methodology: noise and sound measurement
- Methodology: heat tests
- Test rig
- ComputeBench (OpenCL)
- ComputeBench (CUDA)
- SPECviewperf 2020 and SPECworkstation 3
- FLOPS, IOPS and memory speed tests
- 3D rendering 1/2 (LuxMark and Blender@Cycles)
- 3D rendering 2/2 (Blender@Radeon ProRender a Eevee)
- Photo editing
- Broadcasting
- Password cracking
- GPU clock speed
- GPU and VRAM heating
- Net graphics power draw
- Analysis of 12 V subcircuit power supply
- Analysis of 3.3 V subcircuit power supply
- Noise level
- Frequency response of sound
- Conclusion
