[Gamegpu.ru]Call of Duty Modern Warfare Remastered GPU Perf

[antihelten]

Yes I am not suggesting that it is the holy grail of game testing, but that's as far as I can go as a hobbyist benchmarker. I just believe it is better to use it, than not to and just rely on just framerate averages/mins/maxes.

It does expose some shortcomings very nicely however. Like with what I was seeing in Shadow Warrior 2. I felt there was something wrong but I couldn't put my finger on it, until I saw the accumulated data. On the other hand in my ROTTR testing it also highlighted nicely the very good experience I was getting and so on.

Actually that's why I use gameplay video/msi graphs/fraps scores/flac analysis to show the performance. It's the combination of all those that gives an approximation of where things stand. It's not perfect, but it's ok from my point of view.

It's actually fraps that I am starting to fear. It's getting old. Development has stopped, the dev does not respond and game engines progress rapidly. It's a matter of time before it will not be able to collect usable data I'm afraid. I have sent Mirillis an email with some suggestions regarding their Action! program. I hope they will listen.

What kind of raw data you mean? The csvs?

I definitely agree with you that it's better than average, and vastly better than mins/maxes. Personally though I still think 90/95/99 percentile scores are preferable, since one they are easier to interpret and two they are arguably the most important metric for experienced smoothness (i.e. experienced smoothness is dependent upon the marginal frames, question is simply at what percentile we define the margin as significant).

Regarding exposing shortcomings I agree that it can do quite nicely, but looking at those graphs I would expect the 970 to feel roughly comparable to 60 FPS and the 7950 to feel roughly comparable to 30 FPS, however the derived scores are 90 FPS and 35 FPS. As such it only appears to accurate in one of the two cases. I don't think this is necessarily a problem with the scoring scheme used however, more likely it's a problem with the graph and the granularity of it, it can often be difficult to determine exactly how common spikes actually are when you clump some 60,000 data points together.

Since graphs can often be a bit misleading in this manner, I personally think it's useful to do a bit of sorting of the data to get a better feeling for it, and it was basically for this that I asked if you had the data in raw format, and yes csv would be fine.

 

[psolord]

I see what you mean. Not enough granularity on a 1920*1080 pic to show a 15min benchmark run. I understand that not all frames can be equally timed, when running with a non vsync, max speed mentality, in order to benchmark.

Do you think if I posted all FLA graphs like so, it would be more informative. I decided not to in order to save space in the thread.

Spoiler: 970 graphs

Also you can get the 970's csv here.

edit PS I just saw that we can add spoilers in the threads as well, epic, lol

 

[antihelten]

I see what you mean. Not enough granularity on a 1920*1080 pic to show a 15min benchmark run. I understand that not all frames can be equally timed, when running with a non vsync, max speed mentality, in order to benchmark.

Do you think if I posted all FLA graphs like so, it would be more informative. I decided not to in order to save space in the thread.

Spoiler: 970 graphs

Personally I think graphs like these two are much more informative than the previous frame time graphs. From this it is quite evident that the frame hump starts quite early at about the 86th percentile, where preferably you wouldn't want it to start until the 95th percentile or so. This would indicate that the hump is more prevalent that normal, i.e. affects more frames than usual.

The histogram also clearly shows a secondary peak at around 14-15 ms, and given that this is twice the frame time of the main peak (at 7-8 ms), this would be quite noticeable.

Of course with all this we're still only talking about a game dipping down to 60'ish fps from 130 fps, so it would still be perfectly playable. But if the absolute frame rate numbers were lower, then a relative drop of this magnitude would be very noticeable.

Also you can get the 970's csv here.

edit PS I just saw that we can add spoilers in the threads as well, epic, lol

Seems like that download site attaches a trojan to the file (at least that's what windows defender tells me), if possible you could just throw the numbers into a google sheet and share the link to that.

 

[psolord]

Ok I will be posting the whole group of the graphs from now on!

Windows Defender thinks even the air we breathe is contaminated with trojans!

I am not familiar with google docs so here is a mega link

 

[Head1985]

GTX970 oc vs 1070 oc in one test scene....

1920x1080 same settings like in video
GTX970OC 100fps
https://youtu.be/kF1xZqApJKU?t=75
1070OC 140Fps
https://abload.de/img/h1_sp64_ship_2016_11_bnkdj.png
It is little cpu bottleneck with GTX1070(95% GPU usage).I have 6700k 4.5Ghz and 3Ghz memory cl14(Thats why only 40% difference)

then i swith to 1440p and 110fps(still 10% above 970 in 1080p)
https://abload.de/img/h1_sp64_ship_2016_11_5qjve.png

 

[Head1985]

Some more testing vs that GTX970.First scene in 1080p 1440p and 4k just like in his video.
I am again dissapointed by GTX1070 performance.In 1080p and 1440p its bellow 50% gain.Only in 4k(probably because memory bug on GTX970) its 55-60%.Nowhere near 84% gain in 4K in gamegpu benchmarks...
GTX970 gain alot more from oc thats why in gamegpu benchmarks there is 55-60% gap in 1080p and1440p and 84% in 4k(both cards stock).But After both cards OC the gap is bellow 50%.

 

[Head1985]

I see what you mean. Not enough granularity on a 1920*1080 pic to show a 15min benchmark run. I understand that not all frames can be equally timed, when running with a non vsync, max speed mentality, in order to benchmark.

Do you think if I posted all FLA graphs like so, it would be more informative. I decided not to in order to save space in the thread.

Spoiler: 970 graphs

Also you can get the 970's csv here.

edit PS I just saw that we can add spoilers in the threads as well, epic, lol

Hey are you interested in some GTX 970 vs 1070 benchmarks?I have send you Pm.

 

[psolord]

I am using quite enlarged view in this forum,in order to make the side panel go away and with it the top bar also goes away, so I cannot see any notifications, lol. I will have to report this I guess.

As for your request, I will see to it tomorrow.

 

[psolord]

Hey head1985.

Here is your benchmark result. Benchmark started at exactly where Captain Price appeared and was set for 51secs. Didn't touch the mouse.

2016-11-21 16:22:53 - h1_sp64_ship
Frames: 6652 - Time: 51000ms - Avg: 130.431 - Min: 112 - Max: 146

GTX 970@1500/7600.

These are the settings I used. Resolution 1920X1080.

Spoiler: benchmark settings

I only have a 1080p screen however. Do you want me to use DSR to emulate 1440p and 4k?

 

[Head1985]

Yeah use DSR for 1440p and 4k.
Btw memory is at 7000 or 8000?You should oc memory at 8Ghz

 

[psolord]

Ok here are all of them.

Correction clocks are 1500/7600.

2016-11-21 16:22:53 - h1_sp64_ship
Frames: 6652 - Time: 51000ms - Avg: 130.431 - Min: 112 - Max: 146 1080p

2016-11-21 16:53:17 - h1_sp64_ship
Frames: 4591 - Time: 51000ms - Avg: 90.020 - Min: 79 - Max: 98 1440p

2016-11-21 16:55:08 - h1_sp64_ship
Frames: 2462 - Time: 51000ms - Avg: 48.275 - Min: 43 - Max: 54 4k

 

[Head1985]

OK i will benchmark it in same settings

 

[Head1985]

Ok first stock GTX1070 at 1870/8000

2016-11-21 16:30:41 - h1_sp64_ship
Frames: 9837 - Time: 51000ms - Avg: 192.882 - Min: 172 - Max: 216

2016-11-21 16:31:55 - h1_sp64_ship
Frames: 6848 - Time: 51000ms - Avg: 134.275 - Min: 120 - Max: 147

2016-11-21 16:33:06 - h1_sp64_ship
Frames: 3629 - Time: 51000ms - Avg: 71.157 - Min: 63 - Max: 80

And GTX1070 oc 2114/9400

2016-11-21 16:25:55 - h1_sp64_ship
Frames: 11041 - Time: 51000ms - Avg: 216.490 - Min: 193 - Max: 244

2016-11-21 16:27:10 - h1_sp64_ship
Frames: 7782 - Time: 51000ms - Avg: 152.588 - Min: 136 - Max: 170

2016-11-21 16:28:38 - h1_sp64_ship
Frames: 4160 - Time: 51000ms - Avg: 81.569 - Min: 73 - Max: 90

i didnt expect that big gap.How about cpu bottleneck?

 

[psolord]

Awesome! I may get a 1070 myself too! Vega seems to be very late and I'm bored.

My GTX 970 was at 97% load in 1080P and 99% load in 1440p and 4k.

Scores seem normal to me.

192.882 / 130.431 = 47.6% faster

152.588 / 90.020 = 68.8% faster

81.569 / 48.275 = 68.7% faster

isn't that how the delta is normally looking?

I've seen the 1070 being up to 80% faster in some games. Forza Horizon 3 I think and a couple others.

 

[Head1985]

I think 68% is too much.On pcgameshardware they testing aftermarket cards and gap is usually 50-55%.I think your 2500K holding you back.

 

[psolord]

I have posted my normal benchmarks along with the gpu usage in post#14 of this thread. The 2500k is very good at keeping the gpu maxed out even at 1080p. The gpu was running at 97% there as well. It is a game peculiarity. It's steady at 97%.

If you have a cpu limit, the gpu usage will be all jagged up, ranging from say 60% to 90% and going up and down all the time. That's not the case here.

The reason the higher resolutions have a higher delta, is because the 970 hits a fillrate wall much sooner than the 1070 does.

Also as you can see your 1070 is not 68% faster than the 970 at 1080p. That indicates that maybe the 1070 is actually cpu limited in that res and not the other way around.

For the 1070 to be 68% faster than the 970 is not unheard of.



Seems to becoming more common in newer games too. Mafia 3 had a 80% delta, before the 1.040 patch came out. Heck it may still be that big. I haven't seen deltas with the 1.040 patch.

 

[Head1985]

Well its 68% in all resolutions(65% in 1080p 1070oc vs 970oc).You comparing stock 1070 in 1080p.I dont know what other reason is behind that gap.If you look at this video he using filmic SMAA2x (which have around 10% performance impact vs SMAA1x) and still have higher performance on his GTX970 than you.He also using 6700k 4.6Ghz.(yeah he have 970 on 1555Mhz, but 1555 vs 1500 is only 3.6% difference in clock)

 

[Head1985]

I have posted my normal benchmarks along with the gpu usage in post#14 of this thread. The 2500k is very good at keeping the gpu maxed out even at 1080p. The gpu was running at 97% there as well. It is a game peculiarity. It's steady at 97%.

If you have a cpu limit, the gpu usage will be all jagged up, ranging from say 60% to 90% and going up and down all the time. That's not the case here.

The reason the higher resolutions have a higher delta, is because the 970 hits a fillrate wall much sooner than the 1070 does.

Also as you can see your 1070 is not 68% faster than the 970 at 1080p. That indicates that maybe the 1070 is actually cpu limited in that res and not the other way around.

For the 1070 to be 68% faster than the 970 is not unheard of.



Seems to becoming more common in newer games too. Mafia 3 had a 80% delta, before the 1.040 patch came out. Heck it may still be that big. I haven't seen deltas with the 1.040 patch.

If you want compare these cards then compare aftermarket cards and not GTX970 at 1200Mhz vs GTX1070 at 1870mhz(yes reference 1070 boost at 1800-1900mhz thats why they have crap oc)
http://www.pcgameshardware.de/Mafia-3-Spiel-13148/Specials/Technik-Test-Review-1210010/
51% at 1080p
48% at 1440p
53% at 4k
This is why i want compare always OC vs OC and not reference cards.Also this is why i think you are cpu bottleneck because that huge gap.It should be alot less if we compare max OC GTX970 vs max OC 1070.Because maxwell cards have alot more oc headrom than pascal.As you can see i gain only 13% from OC with GTX1070.With GTX970 you gain 20-25%(from reference at 1200Mhz) so the gap must be lower.
Before GTX1070 i was runing GTX970 1500/8000 and that card have GTX980 performance.

Even if afterburner shows 99% GPu usage you can still be cpu/platform bottleneck.I know this i upgraded from 2500k 5Ghz to 6700K 4.5Ghz

 

[psolord]

My GTX 970 is boosting at 1400Mhz, so the 1500Mhz I am getting is not much of performance gain. But I am not complaining, it's a great card. I am looking at Gamegpu's benchmarks mostly. I'd like to think that they are a serious website and they are using equal tiered cards and not stock old ones and OC new ones.

Now to get back to the matter at hand, the fact that my 1080p shows lower delta compared to your 1070 at 1080p, does not show that I am cpu limited. It shows that you are.

Lets see them again.

192.882 / 130.431 = 47.6% faster

152.588 / 90.020 = 68.8% faster

81.569 / 48.275 = 68.7% faster

In order for that 47.6% to be equal to the other percentages of 68% my framerate should have been 114fps. But it is not. It's 130fps. So how am I the one with the cpu limit?

Other than that, you are going from 192fps for 1080p, to 152fps for 1440p. You are getting 26% less performance, for 77% increase in resolution. Does that sound ok to you?

I on the other hand, am getting 130fps for 1080p, and 90fps for 1440p. So I am getting 44% less performance, for 77% increase in resolution. Which one of the two, sounds like the most probable cpu limited scenario? Come on my friend, do the math.

Now regarding the performance of a gpu going up, if you put it on a faster cpu, well I honestly don't know what to say to that. What you are essentially saying, is that if for example I have 100% gpu usage for 100fps on my 2500k, if I get a 6700k I will somehow be getting say...110fps for 100%? No. That's not possible. 100% is 100% no matter what cpu you use.

Don't forget that I have done extensive testing on my 970 in three cpus (see my signature). In all the tests that I was getting the same result, the gpu usage was maxed out. When the gpu usage was maxed out, I was getting the same performance, whether the cpu running the card was the Q9550 or the 2500k.

See for example my Tomb Raider test or Unigine Heaven. Same 100% gpu load on the Q9550 and the 2500k, same performance. Now see Thief or Watchdog, to see what a cpu limit manifests in the gpu usage graph at the end of the video.

Another member in this forum told me that before. This is not happening. Maxed gpu usage means maxed out gpu usage. End of story. You cannot get more framerate on a better cpu. At best you can get an improvement in frametimes. That is if all else is equal. No different game versions or drivers that do affect performance on a specific game (most drivers don't-unless specifically mentioned in the release notes).

I also happen to have done a gazzilion tests on my 5850/570/7950 on all my cpus. No matter where you install the poor 5850, you are still going to get the same performance on the higher tier cpus, if its gpu usage was maxed out on the lower tiered ones. Same for the rest.

Now seriously, have you checked your gpu usage at your 1080p run of Modern Warfare remastered? Is it maxed out? I am intrigued.

 

[Head1985]

Again you comparing stock GTX1070 in 1080p.I have 216Fps with 1070 OC.
gap is 66% in 1080p
68% in 1440p
68% in 4k

Btw yes i have seen in some games where with 2500k and 99% gpu usage i have lets say 50fps and with 6700k its 60fps or more.

 

[psolord]

Oh I see. I copied the wrong numbers scrolling up. My bad.

We are good then. Everything's perfect!

Ok I have never seen 100% gpu load meaning a different result on another cpu, but let's leave it for another time!

 

[Head1985]

its not about 100% GPu usage.There are just platform bottlenecks that even if you think your GPU works at max its not true.I have seen this in moded skyrim..With 2500k 4.5ghz i have like 98% GPU usage and around 45Fps.
In same scene with 6700k 4.5Ghz i have 60fps and 99% GPU usage.

 

[psolord]

I had also included Skyrim (slightly modded) in my benchmark suite. It can be heavy on the cpu, that much is true.

I had gotten 130fps on my 2500k and 85fps on my Q9550.

The problem here however, is that their respective gpu usage graphs, were two worlds apart.





See how the gpu usage varies in the Q9550 system and how more solid it is on the 2500k. This is how a better cpu helps get better performance.

Now I am not saying you are crazy and you don't know what you saw. There could some peculiarities in some games that I have not come across. I test a lot of stuff but I am not a benchmark God, lol.

Don't forget that modded games can be tricky, when it comes to benchmarking. You make have installed a newer version of a mod, that does wonders regarding the peformance, and you thought it was the cpu, or something like that.

Have you kept any screenshots, with an OSD, of that performance difference?

This is getting a little of topic however, since this is the Modern Warfare Remastered thread, but cpu+gpu collaboration is always relevant in the grand scheme of things, I guess.

 

[antihelten]

Ok I will be posting the whole group of the graphs from now on!

Windows Defender thinks even the air we breathe is contaminated with trojans!

I am not familiar with google docs so here is a mega link

Cool. I did a quick frame variance graph, and we can see largely the same pattern as for the frametime graph.

This graph shows the frame to frame variance, and thus stutter. Generally speaking you'll want to stay below 2-3 ms, up until at least 95-99th percentile. CoD goes above this at roughly the 75th percentile, which is way too early and worse it goes up 7 ms variance at the 99th percentile (for comparison look at this PCPER review, it contains examples of good low stutter games, and examples of high stutter games as well). All in all this is a pretty bad case of stutter, although perhaps not world ending, but of course everyone's threshold is different.

 

[psolord]

Thank you for your effort my friend. It does seem that the game has some issues with frametimes.

Yes I am familiar with PCperspective's work. I consider trying that method myself, but my grabber records in H264 and fcat needs uncompressed frames afaik. Anyhoo, I will have to make do with fraps data capturing. It may not be perfect, but when there's an issue, it seems that it's capable of illustrating it.

Now don't forget that my runs are pretty long, not limited to 60secs, and when you move about a game engine, chances are that some different loads will pop up from scene to scene. Also my tests force the system to stream a lot of data internally and my benchmarking captures a lot of the system strain. Even an HDD access can produce a spike of 500ms sometimes. I try to avoid that however, because it throws the graph's granularity out of the window.

I am intrigued as to why this specific game seems to have such variance between frametimes, so I did another test. I started the Charlie don't surf mission, and I benchmarked the first 60seconds without touching anything.

Here are the results I got.

Spoiler: 970,2500k, Charlie Don't Surf, 60secs, no Vsync

If you look at the frametimes graph, things start going south at about 19secs. That's about the time that the helicopter reaches land and some stuff other than deep blue begin getting rendered.

Out of sheer curiosity I did the same test, but this time with vsync enabled. Now even with vsync, some frames got some big frametimes still.

Spoiler: 970,2500k, Charlie Don't Surf, 60secs, Vsync

At this point I understand that I am overtesting things, but I'd like to know why this happens and what hardware combination alleviates this. Truth be told. These things are virtually imperceivable. So I think I am going to ask a friend a favor...!