In the present decade, games tend to offer a more realistic environment thus pushing corresponding computer hardware to its limits. Here comes DLSS to the rescue. Many people might have heard about Cyberpunk 2077 or Microsoft Flight Simulator is almost impossible to play without DLSS. Let’s now take a deep dive into the working of DLSS and its applications.
The Future of Gaming – What is DLSS?
Artificial Intelligence is revolutionizing gaming – from in-game physics and animation simulation to real-time rendering and AI-assisted broadcasting features.
DLSS or Deep Learning Super Sampling is an image up-scaling technology developed by Nvidia and is exclusive to GeForce RTX GPUs.
Deliver Us the Moon
DLSS uses artificial intelligence and machine learning to produce an image that looks like a higher-resolution image, without the rendering overhead.
But first, we need to ask, why do we need DLSS and is it better than native rendering?
The issue with native rendering
The concept of native resolution is becoming less and less relevant in the modern era of games and instead, image reconstruction techniques are coming to the fore. The idea here is simple: in the age of 4k displays, why expend so much GPU power in painting 8.3m pixels per frame when we can avail the benefit of processing power can be directed at higher quality pixels instead and be interpolated up to an ultra HD output?
Native rendering at 4k takes a lot of GPU power and VRAM which can give bad frame rates.
For example, the recently released game Death Stranding gives 60-70fps on i9-9700KF + RTX 2080Ti whereas DLSS gives 90-100fps on the same configuration with a reasonably better-looking picture without using much power.
As games become more and more realistic, it will be harder to render in native 4k or 8k. This is where DLSS comes in handy.
How does DLSS work?
To train the DLSS network, Nvidia collects thousands of “ground truth” reference images rendered with the gold standard method for perfect image quality, 64x super-sampling. 64x super-sampling means that instead of shading each pixel once, they shade at 64 different offsets within the pixel, and then combine the outputs, producing a resulting image with ideal detail and anti-aliasing quality.
Working on DLSS
They also capture matching raw inputs images rendered normally. Next, they start training the DLSS network to match the 64xSS output frames, by going through each input, asking DLSS to produce an output, measuring the difference between its output and the 64xSS target, and adjusting the weights in the network based on the differences, through a process called backpropagation. After many iterations, DLSS learns on its own to produce results that closely approximate the quality of 64xSS, while also learning to avoid the problems with blurring, disocculation, and transparency that affect classical approaches like TAA.
DLSS has three modes:
- Performance – 70%+ higher FPS, but a tiny bit worse perceptible quality than native.
- Balanced – ~40% higher FPS, and basically looks like native, maybe some things perceptibly worse but others better (like a better sharpness).
- Quality – ~25% higher FPS, and some things perceptibly better than native, almost nothing perceptibly worse than native.
Control (2020)
Better over time…
DLSS will continue to improve over time since it runs through a neural network. The original version of DLSS, i.e., DLSS 1.0 had far more artifacts than the current DLSS 2.0. This allows games such as Death Stranding to produce a much cleaner image than other image reconstruction systems, like checkboard rendering. Now, the main issue is game support.
Cyberpunk 2077
Currently, DLSS 2.0 is only supported by 15-20 games, which is less than the number of games that support ray tracing. DLSS has a simplistic implementation methodology dominated by RTX GPUs. Therefore, it grabs the potential to be employed in a plethora of games over the coming years.
– Article by Aditya Miskin, 2nd year Department of Electronics and Communications Engineering