Discussion AMD Gaming Super Resolution GSR

[DisEnchantment]

New Patent came up today for AMD's FSR

Patent Public Search | USPTO

appft.uspto.gov

https://www.freepatentsonline.com/y2021/0150669.html

20210150669 GAMING SUPER RESOLUTION

Abstract
A processing device is provided which includes memory and a processor. The processor is configured to receive an input image having a first resolution, generate linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generate non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The processor is also configured to convert the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and provide the output image for display

[0008] Conventional super-resolution techniques include a variety of conventional neural network architectures which perform super-resolution by upscaling images using linear functions. These linear functions do not, however, utilize the advantages of other types of information (e.g., non-linear information), which typically results in blurry and/or corrupted images. In addition, conventional neural network architectures are generalizable and trained to operate without significant knowledge of an immediate problem. Other conventional super-resolution techniques use deep learning approaches. The deep learning techniques do not, however, incorporate important aspects of the original image, resulting in lost color and lost detail information.

[0009] The present application provides devices and methods for efficiently super-resolving an image, which preserves the original information of the image while upscaling the image and improving fidelity. The devices and methods utilize linear and non-linear up-sampling in a wholly learned environment.

[0010] The devices and methods include a gaming super resolution (GSR) network architecture which efficiently super resolves images in a convolutional and generalizable manner. The GSR architecture employs image condensation and a combination of linear and nonlinear operations to accelerate the process to gaming viable levels. GSR renders images at a low quality scale to create high quality image approximations and achieve high framerates. High quality reference images are approximated by applying a specific configuration of convolutional layers and activation functions to a low quality reference image. The GSR network approximates more generalized problems more accurately and efficiently than conventional super resolution techniques by training the weights of the convolutional layers with a corpus of images.

[0011] A processing device is provided which includes memory and a processor. The processor is configured to receive an input image having a first resolution, generate linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generate non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The processor is also configured to convert the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and provide the output image for display.

[0012] A processing device is provided which includes memory and a processor configured to receive an input image having a first resolution. The processor is also configured to generate a plurality of non-linear down-sampled versions of the input image via a non-linear upscaling network and generate one or more linear down-sampled versions of the input image via a linear upscaling network. The processor is also configured to combine the non-linear down-sampled versions and the one or more linear down-sampled versions to provide a plurality of combined down-sampled versions. The processor is also configured to convert the combined down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution by assigning, to each of a plurality of pixel blocks of the output image, a co-located pixel in each of the combined down-sampled versions and provide the output image for display.

[0013] A super resolution processing method is provided which improves processing performance. The method includes receiving an input image having a first resolution, generating linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generating non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The method also includes converting the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and providing the output image for display.

It uses Inferencing for upscaling. As will all ML models, how you assemble the layers, what kind of parameters you choose, which activation functions you choose etc, matters a lot, and the difference could be night and day in accuracy, performance and memory

 

[Thala]

Idk what kind of magic AMD is pulling here, but FSR2.1 seems to do pretty darn well in this comparison here: https://imgsli.com/MTI0NTg4/8/1

https://twitter.com/x/status/1567989636441722880

Yup, that is an example, where I can see FSR comes closer to DLSS. Unfortunately, it is way off in other examples.

 

[Saylick]

Yup, that is an example, where I can see FSR comes closer to DLSS. Unfortunately, it is way off in other examples.

It's not just closer, but the textures look much more detailed for FSR2.1 than DLSS, and the texture settings are the same in this comparison. The texture setting was maxed out.

 

[Thala]

It's not just closer, but the textures look much more detailed for FSR2.1 than DLSS, and the texture settings are the same in this comparison. The texture setting was maxed out.

If you look more carefully, there is no more detail, just a sharpening filter in case of FSR. Still i consider this a positive example for FSR.

 

[Frenetic Pony]

If you look more carefully, there is no more detail, just a sharpening filter in case of FSR. Still i consider this a positive example for FSR.

They both use sharpening, FSR's just works better.

 

[Saylick]

If you look more carefully, there is no more detail, just a sharpening filter in case of FSR. Still i consider this a positive example for FSR.

Hmm, is that so? I'm far from an expert here, but it's really obvious to me which one I think is better. The thin lines are just all smudged in the DLSS implementation, while I like how well defined the lines and edges look in the FSR2.1 implementation.

DLSS Quality:


FSR2.1:

 

[Mopetar]

Frankly without a native image to serve as a reference, we can't really say which is doing a better job. A person can legitimately prefer one look over another, but I want to know what's doing a better job in matching the original and what performance cost I have to pay to use it.

Comparing two post-processed images against each other directly like that will invariably come down to preference, and for a lot of people that preference will probably just match their brand of choice. Unless it's done using some kind of blinding it doesn't add much value to a review or analysis.

 

[Panino Manino]

Couldn't find the XeSS topic, so I'm posting here.

Soon we will probably see a Digital Foundry comparing all competing solutions. Death Stranding was updates and it supports everything. DLSS2, FSR1, FSR2 and now XeSS.

 

[LightningZ71]

Videocardz.com has an article up on reviews by Tom's, pcgamer, and techpowerup on XeSS. It seems to do well enough on a lot of hardware, but, as shown in one review set, it seems to like having at least 6GB of VRAM to play with. Also, as is shown by that same slide, your hardware has to have DP4A support to see a gain. They show it on the Vega64 (which doesn't have DP4A support) and a couple of 5XXX cards (RDNA 1 didn't have DP4A support until the GPU in the 5500) where XeSS actually slows down performance. Basically, unless you have a VEGA VII or a 5500, you can't use XeSS successfully on anything before the 6000/RDNA 2 series cards. We don't have any information on the APUs, but, judging by the feature support, the 2000 and 3000 series APUs will definitely not see a gain from it (they are based on Vega 64 CUs) but it is possible that 4000 and 5000 series APUS MIGHT as they are based on VEGA VII CUs, which saw a mild shader model improvement that supported DP4A. The main concern for the Renoir and Cezanne APUs is that their limited memory bandwidth may be an issue, as it their limited compute throughput being so small in general. Also, on 8GB systems, they won't have more than 4GB of VRAM allocated, and may not even get enough in systems with more allocated depending on total memory usage.

 

[Paul98]

I hope we see some comparisons of FSR2.1 vs 2.0 and see if it fixed up the main issues FSR2.0 had.

 

[moinmoin]

I hope we see some comparisons of FSR2.1 vs 2.0 and see if it fixed up the main issues FSR2.0 had.

Those would be? Ghosting appears to be much improved.

 

[Panino Manino]

I hope we see some comparisons of FSR2.1 vs 2.0 and see if it fixed up the main issues FSR2.0 had.

Sega's game Judgment was updates from 2.0 to 2.1, and also included XeSS.
Good opportunity to compare.

EDIT:

• Addition of FSR2.1 Native quality.
  *Native works only when it has the same drawing resolution, adopting only the FSR2.1 Anti-aliasing effect .

It looks like AMD is finally releasing its NVIDIA DLAA alternative, an antialiasing technology powered by machine learning but rendered at native resolutions. This means higher image quality but no performance benefits from resolution upscaling.

 

[moinmoin]

FSR 2.2 is out:

"FSR 2.2 has new logic that aims to reduce “High Velocity Ghosting” – a situation that is particularly common in racing games, where a third-person vehicle moves at speed on a relatively flat surface."

Don’t cross the streams! Bust more ghosts with the source code to FidelityFX Super Resolution 2.2

Today, the FSR 2.2 source code and accompanying sample is available to all through our GitHub repository.

gpuopen.com

 

[GodisanAtheist]

AMD Could Tease DLSS 3-rivaling FSR 3.0 at GDC 2023

AMD could tease its next-generation graphics performance enhancement rivaling NVIDIA DLSS 3, at the 2023 Game Developers Conference (GDC 2023), slated for March 23. While the company didn't name it, its GDC 2023 session brief references an "exciting sneak peek of new FidelityFX technologies"...

www.techpowerup.com

Words.

 

[coercitiv]

Sigh, now we'll get even more youtubers telling us FG is great for CPU intensive walking simulators.

 

[Mopetar]

I wish that fake frames were nothing more than fake news. Anyone know a way off the darkest timeline?

 

[RnR_au]

FSR3 is coming .

https://twitter.com/x/status/1639372157556781057

 

[Mopetar]

Darkest timeline confirmed. I'll be a lot less disappointed if the world super powers start lobbing nukes all over. Not as though a few mushroom clouds are going to worsen the hellscape.

 

[GodisanAtheist]

Eh, I'll keeping buying yesterday's cards to play last week's games at ultra settings 1440p/144hz.

When fake frames and such are the defacto method of reaching 144/75/60/30 whatever FPS then it's time to really panick.

 

[JustViewing]

Yay!! for fake frames. Next generation battle will be based on 3x,4x,5x.. fake frames... The winner will be who ever generating more fake frames

 

[Timorous]

Let's revive an old meme.

Yo dawg yo, I heard you like fake frames so I turned on motion interpolation and FSR 3 so your fake frames can generate fake frames.