On with the dreaded Fisking:
Originally posted by: ketchup79
nVidia, to my knowledge, has never done anything like this in the past.
1) Splash screen "cheat" with 3DM01SE. 2) 42.63 drivers scored relatively poorly in 3DM03. nV immediately issued a new driver, 42.68, and urged reviewers to bench 3DM03 with it. It raised scores dramatically, but lowered IQ obviously (Edit: remember the GT4 sky? That wasn't the only obvious IQ defect. [ H ] noticed imperfect textures in GT3, and the B3D forums have pics of poor IQ in 3DM03's own IQ test. Hardware.fr also has pics of the imperfect kettle. The FX5800 obviously had and still has trouble with FP).
Even though that is nothing to go on, the facts of the matter make me wonder how objective Futuremark is being.
Why aren't you wondering how objective nVidia is? They seemed to love 3DM01SE, even using it on their site to test your system to see if you should upgrade to one of their new cards (I think this was removed, for obvious reasons).
...
But, looking at what we do know, I am a little curious as to what futuremark is up to:
1. ATI pays Futuremark $50 thousand a year to "Beta test" 3dmark03. nVidia does not. Now, that means Futuremark
has $50 thousand in incentives to make ATI look better. Not saying they do, but the incentive is there.
nVidia was also a Beta member for 3-6 months, I suppose until it seemed they could not convince FM to redo 3DM03 to show their cards in a more favorable light. The incentive is for nVidia to make their hardware similar to ATi's R300 series, as MS chose the R300 as the DX9 template (just as MS chose nV's GF3 series as the DX8 template). Besides, 3DM is a strict DX benchmark, not a bench of how much you can optimize for each unique architecture. If you're going to use 3DM, you should at least know what it represents. It's not meant to show each card in its best light, but to show it in DirectX's light.
2. ATI has had an unspecified amout of time to optimize their cards/drivers for 3dMark03 before the benchmark was released. So any optimizations ATI made for this benchmark would be hidden to the consumer. nVidia could not optimize their drivers for this benchmark until after it was released, so any optimizations made would be very obvious.
As did nVidia, as they were a Beta member for a while. It seems they weren't twiddling their thumbs the whole time. ATi's optimization was admitted, in their press release today, where they said the changed the order of some shaders in GT4 to better suit their cards. This did not affect IQ, nor was it as blatant a cheat as nVidia's various ones, but FM considers it a cheat nonetheless, and ATi has said they'd remove it from future drivers.
3. Futuremark has accused nVidia of cheating. Futuremark has a purely synthetic benchmark, one which it seems can be easily manipulated. nVidia is the #1 manufacturer of 3d Accelerators in the world, and has been since even before Madonion was founded. Why does everyone assume Futuremark is being honest, and nVidia is lying? Is it unreasonable that since nVidia has seniority as an upstanding company, Futuremark could be the ones lying, for the reasons stated in #1?
FM answered your objections quite well in the FAQ they released (the PDF). nVidia does NOT have seniority as an upstanding company; if you knew their history, you'd understand why some people dislike them so much, even though they clearly have had the best hardware for a few years.
It seems people are more concerned with money and less concerned with honesty. I personally don't believe it is possible nowadays to have an unbiased benchmark, especially one that is purely sunthetic.
We're dealing with businesses--they're ALL concerned with money. There are many of us concerned with honesty, though, and some companies may realize honesty is the best policy to engender customer loyalty. As for your unbiased synthetic benchmark worry, read FM's FAQ (PDF) on this issue.
Edit: Argh, [ H ], not without spaces betwixt the brackets.