Discussion AMD Gaming Super Resolution GSR

[DisEnchantment]

New Patent came up today for AMD's FSR

Patent Public Search | USPTO

appft.uspto.gov

https://www.freepatentsonline.com/y2021/0150669.html

20210150669 GAMING SUPER RESOLUTION

Abstract
A processing device is provided which includes memory and a processor. The processor is configured to receive an input image having a first resolution, generate linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generate non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The processor is also configured to convert the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and provide the output image for display

[0008] Conventional super-resolution techniques include a variety of conventional neural network architectures which perform super-resolution by upscaling images using linear functions. These linear functions do not, however, utilize the advantages of other types of information (e.g., non-linear information), which typically results in blurry and/or corrupted images. In addition, conventional neural network architectures are generalizable and trained to operate without significant knowledge of an immediate problem. Other conventional super-resolution techniques use deep learning approaches. The deep learning techniques do not, however, incorporate important aspects of the original image, resulting in lost color and lost detail information.

[0009] The present application provides devices and methods for efficiently super-resolving an image, which preserves the original information of the image while upscaling the image and improving fidelity. The devices and methods utilize linear and non-linear up-sampling in a wholly learned environment.

[0010] The devices and methods include a gaming super resolution (GSR) network architecture which efficiently super resolves images in a convolutional and generalizable manner. The GSR architecture employs image condensation and a combination of linear and nonlinear operations to accelerate the process to gaming viable levels. GSR renders images at a low quality scale to create high quality image approximations and achieve high framerates. High quality reference images are approximated by applying a specific configuration of convolutional layers and activation functions to a low quality reference image. The GSR network approximates more generalized problems more accurately and efficiently than conventional super resolution techniques by training the weights of the convolutional layers with a corpus of images.

[0011] A processing device is provided which includes memory and a processor. The processor is configured to receive an input image having a first resolution, generate linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generate non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The processor is also configured to convert the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and provide the output image for display.

[0012] A processing device is provided which includes memory and a processor configured to receive an input image having a first resolution. The processor is also configured to generate a plurality of non-linear down-sampled versions of the input image via a non-linear upscaling network and generate one or more linear down-sampled versions of the input image via a linear upscaling network. The processor is also configured to combine the non-linear down-sampled versions and the one or more linear down-sampled versions to provide a plurality of combined down-sampled versions. The processor is also configured to convert the combined down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution by assigning, to each of a plurality of pixel blocks of the output image, a co-located pixel in each of the combined down-sampled versions and provide the output image for display.

[0013] A super resolution processing method is provided which improves processing performance. The method includes receiving an input image having a first resolution, generating linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generating non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The method also includes converting the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and providing the output image for display.

It uses Inferencing for upscaling. As will all ML models, how you assemble the layers, what kind of parameters you choose, which activation functions you choose etc, matters a lot, and the difference could be night and day in accuracy, performance and memory

 

[Panino Manino]

If you are always going to do the upscaling in every game (locked console ecosystem) then DLSS is better - it's give both better quality and higher performance.

Yes... and requires a bigger more expensive chip.

 

[GodisanAtheist]

Yes... and requires a bigger more expensive chip.

- And it ties you to the NV ecosystem. Nintendo is a pretty big dog all their own, last thing they'll want to do is hem themselves in with a single provider due to a proprietary software feature.

If anything, I could see Nintendo insisting NV post DLSS over to run on general compute shaders.

 

[Dribble]

Yes... and requires a bigger more expensive chip.

There is a sizable performance hit from doing FSR 2 and it's larger on lower end chips because it's using the shader cores. Tensor cores do not take a lot of die space, it's more efficient to use them. Dedicated hardware is more efficient then general purpose hardware.

- And it ties you to the NV ecosystem. Nintendo is a pretty big dog all their own, last thing they'll want to do is hem themselves in with a single provider due to a proprietary software feature.

If anything, I could see Nintendo insisting NV post DLSS over to run on general compute shaders.

This doesn't make any sense. It's a console you are tied into an eco system anyway, if it's an NV chip you are tied into using NV anyway, all consoles are coded to the metal which is very proprietary.

 

[moinmoin]

- And it ties you to the NV ecosystem. Nintendo is a pretty big dog all their own, last thing they'll want to do is hem themselves in with a single provider due to a proprietary software feature.

Nintendo is going to keep using Nvidia.

Before Nvidia Nintendo's tool chain honestly had been a total mess. Nvidia's main selling point is its software, ecosystem and tool chain.

Nintendo is not a company systematically building up systems and frameworks over time. Instead they like to build fresh stuff as projects and drop them again one or more generations onward. Nvidia offers a solid base to build upon for Nintendo's and third party developers from which I can't see them wanting to move away from.

 

[Panino Manino]

The best investigation thus far.

 

[Saylick]

FSR 2.0 API is out!

AMD FidelityFX™ Super Resolution 2 (FSR 2)

Learn even more about our new open-source temporal upscaling solution FSR 2, and get the source code and documentation!

gpuopen.com

 

[Leeea]

The best investigation thus far.

Comparisons made by Nvidia's marketing department seem questionable.

 

[Panino Manino]

Comparisons made by Nvidia's marketing department seem questionable.

Yes, they'll strive to show all the places FSR 2.0 fails and this is GOOD.

 

[Aapje]

Yes, they'll strive to show all the places FSR 2.0 fails and this is GOOD.

Not if they overlook the weaknesses of DLSS.

 

[Leeea]

Yes, they'll strive to show all the places FSR 2.0 fails and this is GOOD.

Thing is, if you look for a evaluation that was not done by Nvidia's marketing department, you will see they point out places where the latest version of DLSS also falls on its face.

Hardware Unboxed did a good one like this.


Focusing on the problems with one product, while ignoring all other problems of the other product is not:
"The best investigation thus far. "
"GOOD "

It is marketing.

 

[uzzi38]

FidelityFx Super Resolution 2.0 at Cyberpunk 2077 Nexus - Mods and community (nexusmods.com)

Someone created an FSR 2.0 mod for CP2077. Ghosting is a bit of an issue apparently, but I mean, it works

 

[moinmoin]

That's really cool and encouraging that such mods are possible! There are plenty old games that could profit but don't see any official support by their developers anymore. Though the amount of games supporting DLSS through a dynamic library used for this hack is obviously smaller.

 

[uzzi38]

That's really cool and encouraging that such mods are possible! There are plenty old games that could profit but don't see any official support by their developers anymore. Though the amount of games supporting DLSS through a dynamic library used for this hack is obviously smaller.

Every game that supports DLSS in general is potentially open to a similar solution. So I mean, it's not a massive list of games, but it's quite a few times the number of FSR 2.0 native games lol

 

[coercitiv]

DLSS in general is potentially open

Forgive me, I could not resist the sin of selective quoting. I'll see about my penitence now...

 

[Krteq]

FSR2 for Dying Light 2

GitHub - mnxn/DL2FSR2: FidelityFX Super Resolution 2.0 for Dying Light 2

FidelityFX Super Resolution 2.0 for Dying Light 2. Contribute to mnxn/DL2FSR2 development by creating an account on GitHub.

github.com

Mod unavailable

This mod is not currently available

www.nexusmods.com

FSR2 Implementation to DLSS2 titles in matter of hours?

 

[moinmoin]

FSR2 Implementation to DLSS2 titles in matter of hours?

Yeah, as long as DLSS in implemented using dynamic libraries (DLL files) that can be used to hook up FSR 2.0 instead. That's essentially what we were talking about before.

 

[uzzi38]

GitHub - RealIndica/RDRFSR2: FidelityFx Super Resolution 2.0 for Red Dead Redemption 2

FidelityFx Super Resolution 2.0 for Red Dead Redemption 2 - RealIndica/RDRFSR2

github.com

And now Red Dead Redemption as well. Although I don't own the game, so can't comment on quality

 

[Dribble]

This came out a while ago, but RNDA3 is meant to get an AMD tensor core equivalent: https://www.guru3d.com/news_story/tensor_core_equivalent_likely_to_get_embedded_in_amd_rdna3.html

This is no suprise to anyone as the only way they could compete on an equal footing with DLSS is with the same custom hardware.

 

[moinmoin]

This came out a while ago, but RNDA3 is meant to get an AMD tensor core equivalent: https://www.guru3d.com/news_story/tensor_core_equivalent_likely_to_get_embedded_in_amd_rdna3.html

This is no suprise to anyone as the only way they could compete on an equal footing with DLSS is with the same custom hardware.

That's better placed in the RDNA3 thread since FSR 2.0 (which this thread is about) already showed that GPUs don't need tensor cores at all to compete on an equal footing with DLSS. But tensor cores can be useful for other stuff.

 

[DiogoDX]

This came out a while ago, but RNDA3 is meant to get an AMD tensor core equivalent: https://www.guru3d.com/news_story/tensor_core_equivalent_likely_to_get_embedded_in_amd_rdna3.html

This is no suprise to anyone as the only way they could compete on an equal footing with DLSS is with the same custom hardware.

It's just the instructions. No way to know if RDNA3 will have dedicated hardware for that or they will just do on the shaders.

 

[GodisanAtheist]

AMD's MO has always been about doing more with less (or in many cases generalized hardware). Its the Alton Brown GPU design philosophy: never have a single purpose compute unit in your GPU, everything should have at least 2 or more functions at any given time.

I think its just the product of being the underdog and market follower. 10 years of having to be outrageously cost conscious against their competition everywhere will do that to a company.

That's a lot of words to say I'd bet dollars to donuts that AMD is not putting in any fixed function tensor hardware into their GPUs.

 

[Panino Manino]

Version 2.1 is already out there:

It's time to upscale FSR 2 even further: Meet FSR 2.1!

One of our engineers shares the improvements we've made with AMD FidelityFX Super Resolution 2 to ghosting, temporal stability, and more.

gpuopen.com

 

[DiogoDX]

Still waiting for the CP2077 oficial patch.

 

[uzzi38]

Still waiting for the CP2077 oficial patch.

AMD said it's coming soon today

 

[Saylick]

Idk what kind of magic AMD is pulling here, but FSR2.1 seems to do pretty darn well in this comparison here: https://imgsli.com/MTI0NTg4/8/1

https://twitter.com/x/status/1567989636441722880