Considering how many people use a FPS cap 1 below their refresh rate and use V-sync, goes to show a single frame in 60 doesn't disrupt most people, as long as that long frame isn't taking too much of that second (a single frame that is 16.7ms longer).
FPS caps generally work on a frame by frame basis (at least I don't know of any software that uses the smoothed FPS value to calculate the delay to meet the cap value) so not sure why you would think that supports your argument in anyway.
VSync is actually one of the most important reasons for why you want to measure FPS on a frame by frame basis. Say for instance that your game is constantly oscillating between 59 FPS and 61 FPS. Using a smoothed FPS value you would see your game running at a constant 60 FPS and would thus think that VSync would also run at 60 FPS, but in reality VSync will constantly switch between 30 and 60 FPS, thus producing horrible stuttering. This can only be caught if you measure FPS on a frame by frame basis.
A single frame that is 7ms longer than the rest of your FPS doesn't even show up on most peoples radar (your example going from 28 FPS from 40 FPS based on a single frame).
First of all the difference between 28 FPS and 40 FPS is 11 ms not 7 ms (35.7 ms vs. 25 ms, although it's worth noting that the final 99th percentile score was 34.349 ms corresponding to 29 FPS). And are you seriously claiming that people wouldn't notice going from 40 FPS down to 28 FPS, because I very much doubt anyone would agree with that, we're talking about a 30% dip in performance after all.
Obviously if it's only a single frame then it probably wont be too noticeable, but no one's talking about a single frame, we're talking about multiple frames dipping down to 28 FPS over the entire benchmark/game session (in the benchmark in question roughly 100 frames would be at 28 FPS or worse).
EDIT:
But a single frame that lasts 100ms probably would disrupt a user's experience. And having an average of 28 FPS over the course of a second probably would too. That's why both things are tested.
Testing on a frame by frame basis tests both of these things, so there is no need for calculating a smoothed FPS value.
As I said before testing on a frame by frame basis is strictly superior to smoothed FPS values, since it achieves everything the smoothed value does and more. It will show long lasting dips down to 28 FPS, and also short dips down to 10 FPS (100 ms).
Even in many FCAT reviews, they explain that small variances are showing good results, and large ones are bad. I think Tom's Hardware tried to place a value on when it was bad and not so bad.
Obviously some dips will be worse than other I have never said otherwise. But just because some dips wont noticeably affect smoothness doesn't mean that we shouldn't test for dips altogether, which is what you do when you look at smoothed FPS values.
And it's not like you somehow can't tell a small insignificant dip from a large significant dip, which is of course also why sites like THG can even categorize them in the first place. So there is never any harm in testing on a frame by frame basis.
Ok, so you can shut up already.
No ones forcing you to have a discussion on this discussion forum. If it's too much for you, then just walk away from the keyboard.
From a technical standpoint, the minimum is 28 FPS. From a practical stand point, the minimum was 40 FPS.
How in the world are you determining that the minimum was 40 FPS from "a practical stand point"?
The Civ benchmark reports the average frame rate, the 99th percentile frame rate and a frame rate graph, so where in the world are you pulling this 40 FPS number from, and why is that number the "practical minimum"?
And god help us if reviewers ever started showing a single frame measurement as their minimum.
Reviewers are already reporting a single frame measurement as their minimum, since that how many (if not most) in game benchmarks measure it.
But again I don't see how that's relevant here, I never said that reviewers should report minimums (it's a largely useless metric), but should instead use 99th or 95th percentile scores (which is exactly what the Civ 6 benchmark does).
So many people get so hung up on numbers, and prevents them from playing the game if something appears it could be a problem, even when there isn't an issue.
Seriously just read the Techreport article I linked. It seems obvious that you don't understand the problems with using smoothed FPS values instead of the raw frame rate times.
And this isn't just about Civ 6 (obviously an isometric turn-based game isn't as sensitive to bad frame rates as say an FPS), but is a more general issue. There really isn't any scenario where using smoothed FPS will give you a more accurate picture of performance than what using non smoothed values (i.e. individual frame times) will, at best it won't be any worse (when your frame rate is very stable and smooth).