3DVagabond
Lifer
- Aug 10, 2009
- 11,951
- 204
- 106
Well, it really is about common sense. Outliers, are just that, outliers. In stats, outliers are removed from the results because it heavily skew the results. Having outliers in the averaging results is actually less beneficial. The rule of thumb is to remove the highest and loweset from both cards when calculating the average. For example, if in one game the performance delta is 60fps but in all other games the difference is around 20FPS, you remove that game from the averaging results because including that game would actually do more harm in informing the readers.
That's why that game should be removed from the average. Sure, the game should still be benched. The reviewer should include it in the review. It should be noted and commented on why a particular game is heavily favoring one card over the other. BUT, it should not be included in the average.
^^^This. Anyone who thinks that these games are just arbitrarily chosen without considering the individual performance when figured in to the overall review is terribly naive. When there are outliers from the overall result they are looking for, they do cull them or drop the tests. No more Eyefinity/Surround testing from TPU. No more FLIR videos? Continue to test at resolutions where these cards will be mostly CPU bound? While all of the time complaining that it takes so long to do such an all encompassing review, blah...blah...blah...