Time is usually the limiting constraint when reviewing a launch card. Reviewers will sometimes get mere days before NDA runs out. If they don't get a review published, they will not get the page impressions because after the launch, interest goes down.
With no "perfect" review site, that's why you read multiple reviews. I wouldn't trust a single review of ANYTHING, be it graphics cards, or automobiles, or wireless service, or restaurant.
Again, this is a time issue, in addition to funding. Most sites receive free hardware for their reviews, and there's only so much free hardware to go around. Few sites make enough money to justify outright buying huge amounts of hardware for all their reviewers.
No, that's not (always) the problem.
You get reviews where they have the cards already tested and results available on that website. Same tests, same drivers etc. But they aren't included on the same graphs.
http://www.guru3d.com/article/geforce-gtx-580-review/10
No Crossfire/SLI for mid-range cards.
http://www.guru3d.com/article/radeon-hd-6850-6870-crossfirex-review/7
Crossfire/SLI results.
Why do I have to look at 2 reviews to see how 2 mid-range cards compare to a single high-end card? They have the results, they just didn't include them all together.
This is inexcusable, unless the drivers just came out after most of the review was already done.
The main instance was TPU, which simply doesn't rebench older cards regularly it seems. Most sites don't have this problem.
It's only an issue of time if you do stupid things like including benchmarks at 1024x768 on a GTX580.
Ignoring non-game benchmarks (e.g. 3D Mark, Unigine etc):
Anandtech gives 30 'lots' of results, or thereabouts (resolution * number of games, 3*10),
Guru3d 3D does 21 (3*7).
Hardocp does 10. (highest playable + apples to apples = 2*5)
Techpowerup: 80 (5*16) (but no re-benching of older cards).
ABT: 46 (2*23)
You would think a site like Hardocp or Guru3D, which is doing less benching, would have time to spend on something else, such as IQ analysis, or comparing performance with different AA levels, maybe, or just more games! Assuming all benching takes roughly the same amount of time.
Many of these sites don't even bench many cards, e.g. only 3 cards shown for Hardocp, or they already have many cards benched recently (e.g. most sites just re-used the HD6800 review figures because they were still recent and valid, so only GTX580 numbers were required in addition).
What if they ran at different custom settings ignoring presets... and then we'll have someone complaining here that they don't run in the preset which they use. :sneaky: With all the settings possibilities, there is just no way to test all combinations.
I'm not saying test all combinations, just different levels of AA/AF, to see how they impact performance.
Hardocp doesn't really do this, but they do use different settings depending on what's most playable.
But if a card can manage 4xAA at 1920x1200, then why not benchmark with 8x or 16xAA to see if it can handle that? Show us how AA scales. Or even just some testing of morphological AA on HD6800 cards!
Microstutter can be difficult to pin down. I've played with a lot of GTX 295 quad-SLI setups and some of them had it and some didn't. These were all using the same reference cards on the same two motherboards. Some did and some didn't. How are you going to "review" that?
It's not just microstutter, it's other things like what problems were there, did you have to make profiles, what happened with minimum FPS (too many sites don't give minimum FPS numbers)?
LOL. AnandTech tries to do that when possible. See the recent "previews" on upcoming AMD platforms. AnandTech can only do as much as their NDA will allow them to publish. Of course a company can just give out the information earlier, but then we'll be getting reviews of hardware that may not be in the final state which may not be representative of the end user experience (resulting in more forum complaints ). And if hardware (and drivers) are in the final state, what incentive does a company have to not sell stuff that is ready to go? After all, if you're not trying to get ahead of your competition, you're falling behind.
Nope.