A Short study in microstutter

[BrightCandle]

I wanted to show you three pictures of frame time traces from a forum user who only yesterday had a microstutter problem that in this case was caused by a driver fault where the clock speed would drop during the game. An upgrade to Catalyst 12.11 resolved that problem and smoothed out the gameplay. The orignal thread http://forums.anandtech.com/showthread.php?t=2289212 is available. Many thanks for LessThanDan for providing this information as it gives us a study in a game where we have different amounts of stutter being shown. This is a user who perceived microstutter and thus it allows us to understand how much is too much at least at a granular level (for Dan at least).

Here are the traces and the perception that went along with them:

Really bad, not smooth (Hitman High settings, clock speed fault):

Not smooth but better (Hitman Medium settings, clock speed fault):

Solved, felt smooth (Hitman High settings, clock speed working):

Dans tolerance seems to be swings of 5ms or so and occasional drops to 25fps, both he is happy with. Having tested my own its also 5ms and drops to about 30 fps. What is yours?

 

[thilanliyan]

I see MS is THE hot topic now...

In my experience, I noticed it a bit for the brief day that I used 6950 XFire (and I only tried Metro 2033 so it could have been a combination of lack of performance as well). As for single 6950 and 7950, I have not noticed it at all, but then I do not always play the newest games (playing AC: Revelations right now) so that could have something to do with it also.

 

[Dankk]

Dans tolerance seems to be swings of 5ms or so and occasional drops to 25fps, both he is happy with. Having tested my own its also 5ms and drops to about 30 fps. What is yours?

When you tested yours, was it with the same game (Hitman Absolution)? Sorry for the dumb question, I'm just making sure we're testing the same games here.

Are there any other games you'd be interested in testing and comparing results? Suggest some more games you'd be interested in comparing; chances are I have at least one or two of them in my Steam library. I'd love to compare some more results from other games on different people's setups.

 

[BD231]

Not sure I'd trust a guy who didn't think to update his driver on a brand new game and instead made a thread on it.

 

[Dankk]

Not sure I'd trust a guy who didn't think to update his driver on a brand new game and instead made a thread on it.

In my defense, I was using 12.10, which is the latest stable* driver. Forgive me for not always being willing to use the latest beta drivers, even though it's sometimes for the better and I just don't realize it.

Also, BF3 and Sleeping Dogs are not "brand new" games.

 

[jimhsu]

The hot thing seems to be testing Skyrim these days (though I don't understand how that can possibly be a reliable test, given the dynamic nature of the game). Still, got any graphs on that?

 

[BTRY B 529th FA BN]

What app are you using to measure these results?

 

[Dankk]

What app are you using to measure these results?

FRAPS has a built-in benchmarking tool that can measure frame delivery times, down to a thousandth of a millisecond. It then outputs a .CSV file with a table of the frame times, which can be opened in a spreadsheet program and converted into a visual graph.

I'm not an Excel master, but I should probably teach myself how to graph these for future reference.

 

[VulgarDisplay]

I wonder what caused his GPU clock speeds to change. A bad driver install or something?

Also, would someone install Radeon Pro an run these graphs on a single GPU? I guess I can give it a shot when I get home.

 

[BD231]

In my defense, I was using 12.10, which is the latest stable* driver. Forgive me for not always being willing to use the latest beta drivers, even though it's sometimes for the better and I just don't realize it.

Also, BF3 and Sleeping Dogs are not "brand new" games.

Oh ok, so bad drivers from AMD, gottcha. I thought this was about micro-stutter.

 

[Dankk]

I thought this was about micro-stutter.

It is.

 

[DimmyK]

FRAPS has a built-in benchmarking tool that can measure frame delivery times, down to a thousandth of a millisecond. It then outputs a .CSV file with a table of the frame times, which can be opened in a spreadsheet program and converted into a visual graph.

I'm not an Excel master, but I should probably teach myself how to graph these for future reference.

There is a program that was created few days ago. It does it automatically, you just drop csv frametimes file into it and it plots frametimes, fps and rank. All credit goes to raffriff42. http://sourceforge.net/projects/frafsbenchview/

 

[Dankk]

There is a program that was created few days ago. It does it automatically, you just drop csv frametimes file into it and it plots frametimes, fps and rank. All credit goes to raffriff42. http://sourceforge.net/projects/frafsbenchview/

Excellent! Thanks for the link.

Here's the opening cart scene from Skyrim, on all High settings @1080p, 0xAA, no mods except the official HD texture pack.

Here's the exact same scene, exact same settings, but with 8xAA:

Given that the spikes are in the exact same location, and the benchmark was performed during a scripted cutscene, the spikes are probably just the game streaming in more of the level.

 

[Dankk]

Resident Evil 5, all high settings at 1080p with 0xAA. Pretty smooth.

 

[DimmyK]

Resident Evil 5, all high settings at 1080p with 0xAA. Pretty smooth.

I think this thread might be of interest to you, if you post @ OCN. Nothing extremely scientific, just trying to compare frametimes in repeatable runs.

 

[BD231]

It is.

U and the OP obviously have no idea what micro-stutter is.

 

[Dankk]

U and the OP obviously have no idea what micro-stutter is.

No, I think we have a pretty good idea.

Micro stuttering is a term used in computing to describe a quality defect that manifests as irregular delays between frames rendered by the GPU(s), causing the instantaneous frame rate of the longest delay to be significantly lower than the frame rate reported by benchmarking applications, such as 3DMark, as they usually calculate the average frame rate over a longer time interval.

http://en.wikipedia.org/wiki/Micro_stuttering

 

[BD231]

If you had microstutter youd have a complete pause in gameplay followed by normal frame rates. You had had a driver problem.

 

[BrightCandle]

The 4 traces above are all as bad as the hitman medium settings, is they show a decent amount of microstutter. 10ms swings or jumps like that are not right and you should be experiencing stuttering. You want something like the final trace not the middle one or top one in magnitude.

 

[Dankk]

The 4 traces above are all as bad as the hitman medium settings, is they show a decent amount of microstutter. 10ms swings or jumps like that are not right and you should be experiencing stuttering. You want something like the final trace not the middle one or top one in magnitude.

Really? When doing those benchmarks, both games felt quite smooth (although I guess Skyrim felt a tiny bit janky, but it wasn't too bad). Keep in mind the "Avg. Time" value on those graphs is being skewed by the spikes, in reality it's often much faster than that.

Look more closely at the Resident Evil 5 frame times. Look at the scale on the left, and how zoomed in it is. While there's some micro-stuttering going on, it's occurring within the range of 1-3 milliseconds (!). During normal gameplay, that tiny amount of lag doesn't even register mentally to me at all.

It certainly doesn't feel any worse than Hitman on High settings (after clock fix), or Hitman on Medium settings (before clock fix)... even if the graphs make it look that way. I don't know; maybe we're looking at the graphs differently? Those games were definitely not running as bad as you make them out to be. Perhaps I'm missing something.

 

[Sheninat0r]

If you had microstutter youd have a complete pause in gameplay followed by normal frame rates. You had had a driver problem.

The driver problem caused the microstuttering. How is that so hard to understand?

If you look at the graphs provided, you can clearly see the microstutter, and if you read the thread linked in the OP you can clearly follow the problem resolution steps, including the installation of the new beta driver.

 

[jimhsu]

Ah, the opening scene. Never thought of that. Excellent choice for benchmarking, as it's stable, isn't influenced by any mods (if you have any installed), and completely reproducible.

Excellent! Thanks for the link.

Here's the opening cart scene from Skyrim, on all High settings @1080p, 0xAA, no mods except the official HD texture pack.

Here's the exact same scene, exact same settings, but with 8xAA:

Given that the spikes are in the exact same location, and the benchmark was performed during a scripted cutscene, the spikes are probably just the game streaming in more of the level.

 

[jimhsu]

The definition above doesn't differentiate between plain stuttering and "micro-stuttering" - both have a large instantaneous change in framerate relative to actual framerate (that is, the 2nd derivative of the framerate is large). However, in pretty much all tech discussions (let's exclude this month from consideration), micro-stuttering refers to the specific phenomenon caused by SLI/XFire solutions, while stuttering is a phenomenon affected by other factors (drivers, disk access, CPU use, bad program, etc)

This thread paints a different picture: http://hardforum.com/showthread.php?t=1317582

Why does microstuttering happen?
It's a product of the failure of a multi GPU solution to syncronize properly. Frame syncronization is the act of making sure that the time between frames is identical no matter where you take a measurement. In a single GPU solution one GPU builds the image, and then sends it off to the monitor. It then builds another and sends it off as well. Thus, a single GPU solution does not suffer from microstutter. In a dual GPU solution, two GPUs build seperate images. In Alternate Frame Rendering (AFR), GPU "A" must send its image to the monitor exactly half way between the previous frame from GPU "B", and the next frame (which will be from GPU "B" as well).

Note: 20 milliseconds is .02 seconds, or 1/50th of a second.
In my example lets go with a game running at 50 FPS. A frame is built and displayed by GPU "A". Exactly 20 miliseconds (ms) later GPU "B" must have completed building and displaying its frame. Exactly 20 ms after that GPU "A" must have finished building and displaying the next frame, and so on. Each frame must be displayed in exactly 20 ms after the previous one.

I'm currently working on a flash demo to illustrate the point. When its done I'll export it to .gif and post it here... assuming [H] supports gifs lol.

From there, it seems that microstuttering is more of a synchronization issue - note that the changes in framerate are large, but periodic. Normal stuttering does not exhibit any predictable periodicity. Neither should the observations by TR be described as "microstuttering".

 

[Rikard]

I wanted to run some simple checks using my 7950 and used Heaven for its ability to repeat scenes. At first when I ran with Fraps I saw huge stuttering that would have made a game unplayable. I never saw this before, so I was thinking that maybe it is Fraps itself that is causing the stuttering. The heaven score and FPS results were unchanged however, but I could tell visually that the benchmark was not doing well.

However, I repeated the test (and this time managing to remember to store the frame times in a log file ) and I got nice and smooth visual experience, at least as smooth as you get with those FPS scores. Having seen how my min FPS decreases while my average goes up if I am overclocking too much I tried to see if OC causes microstuttering at the OC setting that I consider my best trade off between average FPS and min FPS, namely 1175/1450/20. As you can see below there are some noticeable spikes in the spectrum, but those were also there at factory-OC levels.

Looking at these plots I find it almost strange that I cannot see any stuttering by eye. Either I am not sensitive to it, or there is something fishy with the measurement.

For the record, Heaven was run at 1080p, everything maxed out, and score was 1267, max FPS 124, min FPS 27.

 

[jimhsu]

Try using the RadeonPro benchmarking tool. If you're using Windows 8, I know that Fraps doesn't play well with certain configurations.

Regarding OCs, check that Powertune is set to +20%.