[TECH Report] As the second turns: the web digests our game testing methods

[f1sherman]

I'm not te guy with double standards here.

Really?

You are swinging between:

MOVE ALONG, NOTHING TO SEE HERE.
Just TR and "cherry picked benches at unplayable settings that that on the top of that don't mean a thing when you play"

and

hey look at these previous-gen NV benchies done 2 years ago by "swindler" TR

 

[Imouto]

It was an issue at the time, although it was an issue with both AMD and Nvidia:

I think the reason it is such an issue now, is because it was a surprise/shock finding. Consensus was generally of the opinion that the 7950 ought to thrash the 660 Ti and thus there was surprise that this didn't happen.

Don't you find quite funny that you posted a graph where the 7950 does as good as the 660 Ti?

 

[willomz]

No I don't find it funny, what I find funny is your assumption that I am trying to talk the 660 Ti up and the 7950 down.

Guess what, I'm neutral, I don't have some ulterior motive to make AMD look bad and neither do the tech report. The link you posted of their earlier review makes it quite clear that they don't mind slagging off Nvidia cards when they perform badly.

The funny thing is that you assume that anyone interested in frame latencies must be some Nvidia fanboy.

 

[SirPauly]

If anything this is proof that tech report has no Nvidia bias whatsoever.

Indeed! TechReport is the site that offered strong opinions on Crysis 2's tessellation.

 

[Keysplayr]

Really?

You are swinging between:

MOVE ALONG, NOTHING TO SEE HERE.
Just TR and "cherry picked benches at unplayable settings that that on the top of that don't mean a thing when you play"

and

hey look at these previous-gen NV benchies done 2 years ago by "swindler" TR

You totally beat me to this EXACT post. Very funny watching this guy downplay the issue with the GCN series and keep plugging spots for previous gen cards. And funnier yet is he doesn't see the irony and self defeat.

Why does he protest so much. Or at all for that matter. Seriously though, at this point I think he should be ignored.

 

[Imouto]

Change your whole bench suite - Check
Go and pick a hell of overclocked card like the 660 Ti AMP versus an almost stock 7950 VapoX - Check
Go and change the OS - Check
Tell everyone that you were kidding in all your previous articles - Check
Make the boring GPU community scream like there's no tomorrow - Check
Make AMD panic - Check

I'm not siding either way, I just find the whole GPU enthusiast community amusing.

 

[willomz]

So out of interest which 7950 would you have picked?

Benchmark suites have to move with the times, when people buy a new graphics card it's usually to go with the latest games. Indeed most of these games are parts of bundles.

They looked into the OS issue in quite some detail.

The rest are just you making a fuss.

 

[Imouto]

Why another 7950 intead the vapox? Why not 2 vanilla cards for both camps?

I agree that bench suites must change but not that much. Take a look at both Skyrim and BF3 benches at GTX 660 Ti launch. They're not even close in the showdown article, day and night bro.

The OS thing is just another step in this stair. They even had to make another article to explain it. Sense? None.

Sloppy? Hell yes!

 

[f1sherman]

You totally beat me to this EXACT post. Very funny watching this guy downplay the issue with the GCN series and keep plugging spots for previous gen cards. And funnier yet is he doesn't see the irony and self defeat.

Strength of his convictions is the funniest part

He sounds genuinely upset that people can't see through TechReport's swindler cherry-picking

Indeed! TechReport is the site that offered strong opinions on Crysis 2's tessellation.

Yes, that TechReport

 

[willomz]

Because most people don't buy reference cards, not the sort of people who read review sites. What they should have done is a comparison of both cards overclocked, that would have been interesting.

So they test in a different area of Skyrim with different drivers. Now I've never played Skyrim, but I guess you do have to go outside sometimes. Why is it invalid to go outside?

 

[f1sherman]

Take a look at both Skyrim and BF3 benches at GTX 660 Ti launch. They're not even close in the showdown article, day and night bro.

You do realize that Skyrim and BF3 received several patches, and NV and AMD released several drivers in that time space, or that AMD breaking perfectly working game is not exactly unheard off?

 

[Imouto]

You do realize that Skyrim and BF3 received several patches, and NV and AMD released several drivers in that time space, or that AMD breaking perfectly working game is not exactly unheard off?

In fact there's no BF3 in the recent article, just kidding.

 

[f1sherman]

OK but how come you have let your conspiracy imagination run wild and have investgated every possible suspect,

except, you know, the most obvious one:


"There is no one single thing for, its all over the place - the app, the driver, allocations of memory, CPU thread priorities, etc., etc"

--Dave Baumann--

 

[willomz]

Back to actual discussion, something that interests me is how the game engine responds to these spikes.

If you have a game at a steady 60fps (16ms frames) and you get a spike, say 50ms. Does the game know and move you 50ms forward, or does the long frame still only move you forward 16ms in game time?

This would seem impossible to know before it's happened, but what about the faster frames that usually follow, that seems more predictable.

If you then get some 10ms frames, are these 10ms in game time, or are they 16ms like average, or does the game think it's going to be another slow frame and show 50ms ahead?

 

[f1sherman]

I think there are spikes, and there are spikes... and then there are dropped frames
So it depends, but I am guessing that long frames due to waiting for CPU are worse in almost every way than when waiting for GPU. And then there are engine differences too.

TL;DR I dont have a clue

 

[VulgarDisplay]

Back to actual discussion, something that interests me is how the game engine responds to these spikes.

If you have a game at a steady 60fps (16ms frames) and you get a spike, say 50ms. Does the game know and move you 50ms forward, or does the long frame still only move you forward 16ms in game time?

This would seem impossible to know before it's happened, but what about the faster frames that usually follow, that seems more predictable.

If you then get some 10ms frames, are these 10ms in game time, or are they 16ms like average, or does the game think it's going to be another slow frame and show 50ms ahead?

I think the PC per article explains what happens without v sync on. The frames get chopped which causes tearing. With v sync I would assume there are dropped frames which the viewer would see as a stutter. The thing to remember is the a 50 ms spike is still more than likely barely visible to most people.

I am just really curious if fixes for the issue will be able to render completely smooth motion at lower frame rates allowing GPUs to work less on brute force and more on efficiency. Could see lower power consumption and less noise with better perceived performance even at lower fps.

 

[Granseth]

I take it therefore you don't see microstutter. That is great you would be more than happy with these cards. But your opinion on whether its there or not isn't very useful because you can't see it. To those that can see it and find it a problem its a severe issue.

If I told you games were perfectly playable at 15 fps you would quite rightly point out you were not happy at that and I was likely very wrong about that. You might say 30 fps is smooth enough, or that anything less than 60 was not smooth. The same argument applies to microstutter. My threshold is about 6ms, which seems on the lower side than most. Yours presumably is much higher, if you have never seen it on Skyrim with a 7000 series card then its above 20ms. Now I personally think its unlikely you couldn't tell the difference between 0 and 20ms microstutter but it doesn't matter because it doesn't bother you.

But just because you think that is OK does not mean everyone does. The subjective feeling of lack of smoothness preceded techreports frame time method by many years. TechReport made the measure objective after of years of subjectively people having issues.

Do you realise you are naked yet sire?

How can your threshold be 6ms. Shouldn't it be 1/120 or around 9ms. And for us with a 60Hz screen 16,7ms should be good enough. I need some help to understand if my assumptions is not correct, but how are you able to detect stutter between frames?

I'm more lucky I guess as I normally don't see stutter most of the time, though when I notice I can agree that its a hell of a problem and makes the game unplayable. But shouldn't it be better to have every frame at 20ms, than an unsteady one with an average og 16ms? I thought that was much of the problem, because if everything is slow its just a bit slow and but if there are an uneven frame-time you get stutter.

 

[willomz]

You are going to see stutter regardless. If you have 3 frames on screen, then one of those is from 50ms ago.

The point I was trying to make is that is it only the 'spike' that causes stutter, or is it in part the fast frames we see after the spike?

Stutter for me is about movement and predictability/smoothness of that movement. It's not just the spike that is off-putting, but the period of adjustment after the spike. Until we back at a steady state and your brain recognises this, the images we see break the illusion of smooth movement.

This is speculation as I agree that a 50ms should be almost invisible, but it isn't to me and many others. So I suspect that what we notice is actually a pattern of several frames not behaving as expected.

 

[Ferzerp]

Back to actual discussion, something that interests me is how the game engine responds to these spikes.

If you have a game at a steady 60fps (16ms frames) and you get a spike, say 50ms. Does the game know and move you 50ms forward, or does the long frame still only move you forward 16ms in game time?

This would seem impossible to know before it's happened, but what about the faster frames that usually follow, that seems more predictable.

If you then get some 10ms frames, are these 10ms in game time, or are they 16ms like average, or does the game think it's going to be another slow frame and show 50ms ahead?

Depends on the actual game, but very, very, very, very very few tie motion to frames anymore (dark souls being one of the few).

 

[willomz]

Do you have some idea how it does work? Is motion done at a set rate e.g. every ms? And then when a new frame is started it works with whatever the latest positions are?

 

[Keysplayr]

Change your whole bench suite - Check
Go and pick a hell of overclocked card like the 660 Ti AMP versus an almost stock 7950 VapoX - Check
Go and change the OS - Check
Tell everyone that you were kidding in all your previous articles - Check
Make the boring GPU community scream like there's no tomorrow - Check
Make AMD panic - Check

I'm not siding either way, I just find the whole GPU enthusiast community amusing.

Oh come now. You're just making it silly at this point. If you listened to the podcast, you'd know that both boost and vanilla versions of the 7950 were tested. And I'm sure you'll try and think up some other senseless reason why the testing was unfair. Don't you hear yourself man? This isn't even about fps and your talking about which card was o/c'd. The 660 got LOWER fps but exhibited smoother gameplay. Get it?

 

[Imouto]

Oh come now. You're just making it silly at this point. If you listened to the podcast, you'd know that both boost and vanilla versions of the 7950 were tested. And I'm sure you'll try and think up some other senseless reason why the testing was unfair. Don't you hear yourself man? This isn't even about fps and your talking about which card was o/c'd.

Do you realize that changing an almost stock card for a stock card doesn't make any sense? That wasn't the point anyway. It was about giving some aura of fairness to the whole farce.

660 Ti Stock = 915/1058 Core 1500 Mem
660 Ti AMP = 1033/1111 Core 1652 Mem
7950 Boost Stock = 850/925 Core 1250 Mem
7950 Vapox = 850/950 Core 1250 Mem

Like doing the testing in W8. It's going to be an irrelevant OS like Vista was. Why they changed the OS for their tests at this point? It has less market share than OSX and won't surpass W7 even in its wildest dreams.

Again they changed their bench suite. If they get back the old benches it would go 50/50 again. Doing fine in old games and having this "issue" in the new ones.

It's not like it was unfair, the point is that it was sloppy.

The 660 got LOWER fps but exhibited smoother gameplay. Get it?

Did we read the same review? Cuz it was something like this:

660 wins: Skyrim, Assa Creed III (7950 was destroyed), Borderlands 2
tied: MoH War, GW2
7950 wins: Hitman, Sleeping Dogs

I don't even...

I'm going back to lurking so you and your focus group can have these forums all for yourselves. You've got your suit, enjoy it.

 

[BrightCandle]

The Intel engineer on beyond3d had a wonderful explanation of what is causing a lot of the pain. If we assume that most of the time the frame rate is limited by the GPU then the buffer from CPU/game world is full and the GPU happily chomps on each in turn. But it also means because the buffer is full the CPU sits there not producing a frame. In this mode both are producing a frame at a consistent period, let's say 16ms.

But if one frame takes a long time on the CPU then the GPU now gets ahead. Until the CPU produces the frames for the buffers it gets to run as fast as it likes until once again the buffer is full. But the GPU is basically taking the same time it always did to render the scenes. So the motion and view from which you see the world speeds up for a few frames, as well as jolting before hand.

Its certainly possible its the GPU rendering that is jumping as well, and that too would knock the CPU onto an uneven motion but presumably only for a single frame.

 

[Keysplayr]

Do you realize that changing an almost stock card for a stock card doesn't make any sense? That wasn't the point anyway. It was about giving some aura of fairness to the whole farce.

660 Ti Stock = 915/1058 Core 1500 Mem
660 Ti AMP = 1033/1111 Core 1652 Mem
7950 Boost Stock = 850/925 Core 1250 Mem
7950 Vapox = 850/950 Core 1250 Mem

Like doing the testing in W8. It's going to be an irrelevant OS like Vista was. Why they changed the OS for their tests at this point? It has less market share than OSX and won't surpass W7 even in its wildest dreams.

Again they changed their bench suite. If they get back the old benches it would go 50/50 again. Doing fine in old games and having this "issue" in the new ones.

It's not like it was unfair, the point is that it was sloppy.



Did we read the same review? Cuz it was something like this:

660 wins: Skyrim, Assa Creed III (7950 was destroyed), Borderlands 2
tied: MoH War, GW2
7950 wins: Hitman, Sleeping Dogs

I don't even...

I'm going back to lurking so you and your focus group can have these forums all for yourselves. You've got your suit, enjoy it.

Ah I worded it wrong. My bad. The "aura of fairness" is in your imagination. It's not a matter of fps. Not at all. It's smoothness. The 660Ti can manage a smoother gameplay experience with LOWER framerates than a 7950. I think the editor of H mentioned that what feels smooth at 40fps on the Nvidia card would require or did require 60-70fps on the Radeon to feel at least a similar level of smoothness.
Anyway, thank you for returning to lurking. All the best.

 

[Hitman928]

The 660Ti can manage a smoother gameplay experience with LOWER framerates than a 7950. I think the editor of H mentioned that what feels smooth at 40fps on the Nvidia card would require or did require 60-70fps on the Radeon to feel at least a similar level of smoothness.

Hmm, didn't see that one, got a link?