Face Refinement, Optical Flow and Super Scale
DaVinci Resolve (Studio) video editing applications are highly optimized for hardware acceleration by GeForce RTX graphics cards. These can dramatically reduce the time of some tasks, turning hours into minutes or, for larger projects, days into hours. We’ll take a look at what exactly this is about in a two-part miniseries dedicated to streamlining work in Black Magic Studio video editors with NVIDIA Ada Lovelace GPUs.
Face Refinement
Face Refinement is essentially augmented reality make-up. It can modify the face of the person in front of the camera in a variety of ways without the need to stand in front of a mirror for a long time. How authentically, we’ll leave that to the judgement of those who understand it more than we do.
For our purposes, we won’t go into the artistic side of things, and we haven’t edited anything on Ms. Amber, from whose video Fall Sweater Plans and Projects, Vacation Knitting, New Yarn and Plans To Use It on the YT channel A Lovely Yarn we’ve borrowed the opening 148 seconds.
Computational performance is required for face analysis, based on which the selected filters are then applied. As the face part of the image changes its position, recomputation occurs continuously throughout the recording. This is necessary to ensure that cosmetic adjustments to the face are rendered correctly, without obvious flaws.
The face mapping begins with an instruction to analyze.
With the overlay layer view enabled, you can see exactly how this feature replicates the main features of the face.
Optical Flow
Slowing down video with low fps? It can be done without lowering the smoothness below the usability threshold. That is, as long as the missing frames are computed. For this experiment, we reached for a 4K video at 24 fps – the LG Snowboarding Demo (downloadable here).
After the speed is reduced four times, in this case to 6 fps, the image is naturally jerky. However, this can be eliminated by deploying Optical Flow, which returns the video back to the original 24 fps while keeping the video slowed down as if it were recorded by a high-speed camera. For the best result, which is also the most computationally intensive, we set Motion Estimation to Enhanced Better.
What we’ll be looking at for this test is the frame rate of the live playback.
Super Scale
The image quality of the videos used so far has always been relatively high, but this time what will be worked with reaches a bitrate of only 2.33 Mbps, and there is a lot to improve in the image of the I Jumped From Space (World Record Supersonic Freefall) video. So it’s a good fit for DaVinci Studio’s “Super Scale”, which, in addition to upscaling the image, also sharpens and reduces noise.
We set the effect of both of these tasks to high, and in a week’s time you’ll see how the differently performing GPUs hold up in a performance comparison.
If you would like to test the performance yourself and compare your setup with our results, here are the output settings. For the QuickSync format, the H.265/HEVC codec and an encoder according to the GPU are used. For NVIDIA graphics cards, it is naturally NVENC.
- Contents
- What is it about?
- Magic Mask, Smart Reframe
- Face Refinement, Optical Flow and Super Scale
- AV1 and HEVC encoding
- Explore: what is the AV1 video format?
