Discussion AMD Gaming Super Resolution GSR

[DisEnchantment]

New Patent came up today for AMD's FSR

Patent Public Search | USPTO

appft.uspto.gov

https://www.freepatentsonline.com/y2021/0150669.html

20210150669 GAMING SUPER RESOLUTION

Abstract
A processing device is provided which includes memory and a processor. The processor is configured to receive an input image having a first resolution, generate linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generate non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The processor is also configured to convert the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and provide the output image for display

[0008] Conventional super-resolution techniques include a variety of conventional neural network architectures which perform super-resolution by upscaling images using linear functions. These linear functions do not, however, utilize the advantages of other types of information (e.g., non-linear information), which typically results in blurry and/or corrupted images. In addition, conventional neural network architectures are generalizable and trained to operate without significant knowledge of an immediate problem. Other conventional super-resolution techniques use deep learning approaches. The deep learning techniques do not, however, incorporate important aspects of the original image, resulting in lost color and lost detail information.

[0009] The present application provides devices and methods for efficiently super-resolving an image, which preserves the original information of the image while upscaling the image and improving fidelity. The devices and methods utilize linear and non-linear up-sampling in a wholly learned environment.

[0010] The devices and methods include a gaming super resolution (GSR) network architecture which efficiently super resolves images in a convolutional and generalizable manner. The GSR architecture employs image condensation and a combination of linear and nonlinear operations to accelerate the process to gaming viable levels. GSR renders images at a low quality scale to create high quality image approximations and achieve high framerates. High quality reference images are approximated by applying a specific configuration of convolutional layers and activation functions to a low quality reference image. The GSR network approximates more generalized problems more accurately and efficiently than conventional super resolution techniques by training the weights of the convolutional layers with a corpus of images.

[0011] A processing device is provided which includes memory and a processor. The processor is configured to receive an input image having a first resolution, generate linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generate non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The processor is also configured to convert the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and provide the output image for display.

[0012] A processing device is provided which includes memory and a processor configured to receive an input image having a first resolution. The processor is also configured to generate a plurality of non-linear down-sampled versions of the input image via a non-linear upscaling network and generate one or more linear down-sampled versions of the input image via a linear upscaling network. The processor is also configured to combine the non-linear down-sampled versions and the one or more linear down-sampled versions to provide a plurality of combined down-sampled versions. The processor is also configured to convert the combined down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution by assigning, to each of a plurality of pixel blocks of the output image, a co-located pixel in each of the combined down-sampled versions and provide the output image for display.

[0013] A super resolution processing method is provided which improves processing performance. The method includes receiving an input image having a first resolution, generating linear down-sampled versions of the input image by down-sampling the input image via a linear upscaling network and generating non-linear down-sampled versions of the input image by down-sampling the input image via a non-linear upscaling network. The method also includes converting the down-sampled versions of the input image into pixels of an output image having a second resolution higher than the first resolution and providing the output image for display.

It uses Inferencing for upscaling. As will all ML models, how you assemble the layers, what kind of parameters you choose, which activation functions you choose etc, matters a lot, and the difference could be night and day in accuracy, performance and memory

 

[CakeMonster]

Ugh, I really hope that the sharpening can be 100% user controlled. Not a fan of the effects of sharpening and that could ruin the whole feature for me.

 

[KeinZantezuken]

Resolution / Upscaling method	Console command	Average FPS (approx near the starting rock)
1440p Native	r.TemporalAA.Upsampling 0 r.ScreenPercentage 100	44 FPS
720p Bilinear (naïve method)	r.TemporalAA.Upsampling 0 r.ScreenPercentage 50	85 FPS
720p TAA	r.TemporalAA.Upsampling 1 r.TemporalAA.Algorithm 0 r.ScreenPercentage 50	81 FPS
720p TSR	r.TemporalAA.Upsampling 1 r.TemporalAA.Algorithm 1 r.ScreenPercentage 50	79 FPS
1080p Bilinear	r.TemporalAA.Upsampling 0 r.ScreenPercentage 75	61 FPS
1080p TAA	r.TemporalAA.Upsampling 1 r.TemporalAA.Algorithm 0 r.ScreenPercentage 75	61 FPS
1080p TSR	r.TemporalAA.Upsampling 1 r.TemporalAA.Algorithm 1 r.ScreenPercentage 75	58 FPS

is there a mistake? Your native sections list command setup:

r.TemporalAA.Upsampling 0
r.ScreenPercentage 100

This is not enough to disable AA completely in UE (r.TemporalAA.Upsampling disables the upsampling in TAA pass), you need either:

r.DefaultFeature.AntiAliasing=0

or

PostProcessAAQuality=0

Can you quadruple check that state of our screenshot for native does indeed represent proper native?

P.S. Kinda ironic how you need to go thruogh lots of hurdles in modern engines to render in native res. I really dont like where it is all going

 

[Gideon]

is there a mistake? Your native sections list command setup:

r.TemporalAA.Upsampling 0
r.ScreenPercentage 100

This is not enough to disable AA completely in UE (r.TemporalAA.Upsampling disables the upsampling in TAA pass), you need either:

r.DefaultFeature.AntiAliasing=0

or

PostProcessAAQuality=0

Can you quadruple check that state of our screenshot for native does indeed represent proper native?

P.S. Kinda ironic how you need to go thruogh lots of hurdles in modern engines to render in native res. I really dont like where it is all going

Thanks. I should have specified that I did only want to disable the upsampling part, not TAA completely (as it get's really grainy). TAA itself is always running in all cases (as it would be for actual games) i just compared upscaling to no upscaling, so to say.

 

[KeinZantezuken]

Thanks. I should have specified that I did only want to disable the upsampling part, not TAA completely (as it get's really grainy). TAA itself is always running in all cases (as it would be for actual games) i just compared upscaling to no upscaling, so to say.

Sure, but the issue is that your actual FPS data for native isnt really correct, TAA, especially newly added Gen5 one, can eat up to ~20 FPS (!) even without upsampling path. I think this approach just skews outlook at performance numbers in general since we dont have proper baseline.

 

[leoneazzurro]

Amd has launched the new Adrenalin version with FSR support today.

 

[zebrax2]

So far reviews seems favorable for FSR with most reviewers indicating that ultra quality seems to being quite near to native resolution in terms of image quality at higher resolutions. It isn't as effective in preserving details at 1080p and below though as there simply isn't enough details at rendered image to scale. It's no DLSS 2.0 but its significantly better than playing at a lower resolution from the looks of it. Reviewers didn't notice any difference in image quality between AMD and NVidia cards running FSR and performance scaling seems to be pretty similar between the brands.

Complaints includes shimering and blurriness specially at lower FSR quality settings, the lack of AAA games at launch and Epic missing from the list of game studio partners.

 

[BFG10K]

It actually works pretty well and is relatively simple to integrate into the game engine, using an open-source shader.

And it works everywhere, including older nVidia cards.

AMD FSR FidelityFX Super Resolution Quality & Performance Review

Finally! AMD has its NVIDIA DLSS competitor ready, and it even works on NVIDIA cards. We're testing the company's new super-resolution technology in five games. The comparison images in our AMD FSR review let you dig into all the finer details, and our performance results cover multiple...

www.techpowerup.com

 

[Leeea]

Wow, it looks like they actually delivered something good!

It appears it will trade blows with DLSS 2.0 in higher quality modes. Lower quality modes are a easy win for DLSS 2.0.

 

[Mopetar]

Also really big for anyone still hanging onto a Pascal card to get through the GPU shortage / mining boom since it gives those cards a solid bump for anyone wanting to stick it out with their current card until the situation improves.

 

[Krteq]

Some other reviews:

https://www.techspot.com/review/2277-amd-fsr-analysis-benchmark/
https://www.guru3d.com/articles-pages/amd-fidelityfx-super-resolution-preview,1.html
https://www.computerbase.de/2021-06/amd-fidelityfx-super-resolution-fsr-test/
https://www.pcgameshardware.de/Fide...DLSS-Test-Vergleich-Benchmark-Review-1374282/
https://www.hardwareluxx.de/index.p...ution-bildqualitaet-und-leistung-im-test.html

 

[coercitiv]

To quote GN: Good first start. Needs more games.

 

[Heartbreaker]

Looks quite weak at Digital Foundry:

 

[Makaveli]

Looks quite weak at Digital Foundry:

Not surprised about what DF has to say compared to others.

 

[Krteq]

Looks quite weak at Digital Foundry:

As expected

Check other reviews and their conclusions, those are pretty positive.

What disappointed me the most in that DF review is comparison with TAAU where they compared it against FSR performance preset only, why?

 

[Leeea]

Looks quite weak at Digital Foundry:

What the nvidia shills have to say is valuable, but it needs to be taken in context.

 

[uzzi38]

Looks quite weak at Digital Foundry:

According to the same guy who claimed DLSS 1.0 is comparable to native.

I couldn't care less for DF reviews for this sort of stuff, especially not from Alex.

 

[DisEnchantment]

According to the same guy who claimed DLSS 1.0 is comparable to native.

I couldn't care less for DF reviews for this sort of stuff, especially not from Alex.

I can't agree more.
With the UE5 demo as well, what was hilarious to me was that Alex had a tough time acknowledging 6800XT is faster than NV's best in UE5 demos. They even went on to ridicule it by putting some meme.
The other guy was Richard with his flailing arms distracting me more than an expressive Italian waving his hands about trying to get his point across.
The rest of the guys are OK.

 

[uzzi38]

Btw if anyone wants to give FSR a try themselves, The Riftbreaker has a free demo/prologue that supports it and is only a 5GB download.

If anyone has a GPU/CPU w/ iGPU not supported by FSR officially it would be pretty cool to see how those work as well.

 

[coercitiv]

Strange that DF cares so much about IQ, what happened to "in the moment performance"?

 

[GodisanAtheist]

I'm too lazy to look if anyone tested, but wondering if these work on AMD's Vega based APUs? My wife is running a Ryzen 2200G chip and this might be exactly what APUs need for that little extra performance boost.

As for a fully fledged desktop, I'm glad to see FSR isn't just a complete turd like many of us feared it would be, but I stick by the same statement I made regarding DLSS: this is great to extend the life of an aging video card, but i desperately hope it does not become the crutch used to justify poor gen to gen native res performance increases as we go forward.

Neither of these features should be necessary on a new card gaming at "common" (up to 4K) monitor resolutions at the top end.

 

[Hitman928]

I'm too lazy to look if anyone tested, but wondering if these work on AMD's Vega based APUs? My wife is running a Ryzen 2200G chip and this might be exactly what APUs need for that little extra performance boost.

As for a fully fledged desktop, I'm glad to see FSR isn't just a complete turd like many of us feared it would be, but I stick by the same statement I made regarding DLSS: this is great to extend the life of an aging video card, but i desperately hope it does not become the crutch used to justify poor gen to gen native res performance increases as we go forward.

Neither of these features should be necessary on a new card gaming at "common" (up to 4K) monitor resolutions at the top end.

Gamer's Nexus tested a 5700G with good results.

 

[DisEnchantment]

Strange that DF cares so much about IQ, what happened to "in the moment performance"?

Ironic to think that this narrative was brought by people who considered the FarCry RT remake and bug fest CP2077 the best form of artistic expression of RT tech and belittling PS5/Insomniac's achievements in R&C

 

[uzzi38]

I'm too lazy to look if anyone tested, but wondering if these work on AMD's Vega based APUs? My wife is running a Ryzen 2200G chip and this might be exactly what APUs need for that little extra performance boost.

As for a fully fledged desktop, I'm glad to see FSR isn't just a complete turd like many of us feared it would be, but I stick by the same statement I made regarding DLSS: this is great to extend the life of an aging video card, but i desperately hope it does not become the crutch used to justify poor gen to gen native res performance increases as we go forward.

Neither of these features should be necessary on a new card gaming at "common" (up to 4K) monitor resolutions at the top end.

A friend also tested on a 4700U, no problems here either.

 

[AtenRa]

I'm too lazy to look if anyone tested, but wondering if these work on AMD's Vega based APUs? My wife is running a Ryzen 2200G chip and this might be exactly what APUs need for that little extra performance boost.

As for a fully fledged desktop, I'm glad to see FSR isn't just a complete turd like many of us feared it would be, but I stick by the same statement I made regarding DLSS: this is great to extend the life of an aging video card, but i desperately hope it does not become the crutch used to justify poor gen to gen native res performance increases as we go forward.

Neither of these features should be necessary on a new card gaming at "common" (up to 4K) monitor resolutions at the top end.

VEGA 64 link (Hardware unboxed)

 

[uzzi38]

Btw if anyone wants to give FSR a try themselves, The Riftbreaker has a free demo/prologue that supports it and is only a 5GB download.

If anyone has a GPU/CPU w/ iGPU not supported by FSR officially it would be pretty cool to see how those work as well.

Just a note for anyone trying Riftbreaker, the TXAA implementation has to be one of the worst AA implementations I've seen in a very long time. Any movement causes huge quality drops even when rendering natively, without FSR toggled on.

I recommend disabling it in the initial-config file