Futuremark has released a new patch and...

[McArra]

Well, it seems they have found that Detonator FX drivers really are optimized for 3dMark 2003 and
have released a patch to avoid this cheat. Now we'll see how GForce FX performs!!!!

 

[stonecold3169]

This is great news.

While people might say "Who cares, it's just a benchmark, ATI still beats them in games, so don't worry about it", well, it doesn't work like that.

The people of anandtech-type computer intellect and desire for new tech make up only less then 3% of all sales. The mainstream consumer MIGHT read one or 2 reviews online of the card before he buys, and if he sees one card beating another, it'll sway them one way or another.

The thing that hurts is when a consumer is swayed by basically false info, and then the other company loses what should have been a valid sale. Better to keep things even and let the consumer see true, every day results. It seems impractical, but I think the best way for reviewers to benchmark would be to pick random titles and programs to use so that it would be impossible to stack it one way or another.

 

[blahblah]

Read PDF here

Read their findins.

It will no doubt suprise some of you that ATi's latest driver also has some irregularity.

 

[Rogozhin]

Good for futuremark not putting up with this shat.

rogo

 

[Rogozhin]

Well

"Our investigations reveal that some drivers from ATI also produce a slightly lower total score on this new build of 3DMark03. The drop in performance on the same test system with a Radeon 9800 Pro using the Catalyst 3.4 drivers is 1.9%. This performance drop is almost entirely due to 8.2% difference in the game test 4 result, which means that the test was also detected and somehow altered by the ATI drivers. We are currently investigating this further."

Looks like maybe (not conclusive yet) that ati might have cheated by 1.9%

And nvidia did cheat (conclusive) by 24%.

Time for a toast to futuremark for leveling the playing field

rogo

 

[vss1980]

The 8% or so difference they find with ATI drivers is not what I would think is down to cheating, but more a case of the drivers optimising the rendering pipelines to get the most speed out of the card as possible for that particular application which is nothing new and not really a cheat. It is usually used to make a card perform things in a way more optimised for that program but can also be used to fix bugs in rendering specific to a certain program.

The GF FX lost out by nearly a quarter of its speed... that is something that will not impress many people and I fully expect nvidia to start bitching about it in some way or another.

 

[gururu]

5900U drops 24.1 % with new 3dmark03 build.
9800pro drops 1.9% with new build.

gaddamn! Nvidia drivrs had eight instructions designed to ignore 3dmark commands.

gururu

 

[blahblah]

Actually, the document goes on to explain why cheating in general is bad.

For those of you who still think that as long as it works in games, you should read the article and see if you agree with the following points.

1. This cheat can be implemented for game benchmarks. (Remember Game bench marks are also static)
2. It will be harder to detect these cheats in Game bench marks the 3DSysMark
3. The driver could disable the cheat when you run the games. So there is the potential of you getting say 85FPS in game bench, but when you run the actual game, you might get 60FPS. Not exactly what you were expecting.

Anyways, it doesn't really matter who is cheating in drivers, the problem is that All reviews of Graphic card is done with either Game Bench or Synethic Bench. In order to compare one card from the next, all these bench mark app has be static. Ie they are the same if you re run them.

With detection cheat, drivers can boost bench mark scores as FutureMark has shown and hence give consumer an inflated value. If I bought a card thinking it will perform at X and got a card that performs 20% less than X, I'd be upset.

Forget this NV Vs ATi debate, both company have done it in the past and both company are probably still doing it. What we need to do is let them know that Cheating is not the right way to get our Hard earned dollars. (Okay some of us anyways)

Don't let them get away with the LAME "we are just showing your how these bench are unrealiable" excuse. If NV or ATi really wanted to do that, they would tell us ahead of time, not after they been found out.

Just my thoughts.

What is a definition of "NVidiot" or "FanAtic", when the truth hits you smack in the face, you still refuses to acknowledge it. My friend, in this case, you are a lost cause, at least when it comes to Video card.

 

[Nemesis77]

Where are those "NVIDIA did not cheat!"-folks now?

 

[kylebisme]

wallowing in a pit of denial i imagine.

 

[McArra]

In the window if they have bought a FX, specially if it is a 5900/5800

jejejeje

 

[Nemesis77]

take a look at http://www.hardware.fr/html/news/#5797! NV's performance drops by as much as 56% in some tests!

 

[gururu]

HardOCP regarding 3dmark fix:

To us, things like this just solidify our belief that 3DMark03 overall score is useless as a real world benchmark.

are these guys stupid? futuremark is trying to improve their bench and hardocp is slamming them harder.
and this is after coming to the defense of nvidia's 'driver optimisations'. I hate it when sites lose sense of
objectivity.

 

[Rogozhin]

That point H is trying to make is irrelevant to what is happening. It's important as a "gauge" in which cheating can be found and exposed so as to keep vga makers from cheating on game benchies.

I would like more premises for that conclusion they reached too.

rogo

 

[Ady88]

yeh thats pretty lame from oftOCP

 

[nippyjun]

I still think that benchmarking actual games makes more sense. Just compare the cards running today's games.

 

[Nemesis77]

Originally posted by: nippyjun
I still think that benchmarking actual games makes more sense. Just compare the cards running today's games.

These cheats could be used in game-benchmarks as well. And fact is, NV cheated. Can you trust them from now on? They explisitly said that they thing 3DMark is worthless and they will not waste their time optimizing their drivers for it. Looks like they flat-out lied

 

[boyRacer]

Originally posted by: Nemesis77
Where are those "NVIDIA did not cheat!"-folks now?

NVIDIA did not cheat!

 

[WetWilly]

Originally posted by: gururu
HardOCP regarding 3dmark fix:

To us, things like this just solidify our belief that 3DMark03 overall score is useless as a real world benchmark.

I think they've just officially certified themselves nVidia w(H)ores.

 

[AmdInside]

I still don't understand why so much emphasis on 3dmark 2003. Personally, I never doubted that NVIDIA and ATI cheat on 3dMark 2003. In fact, I don't even have 3dmark 2003 installed on my PC. While I used to place a lot of emphasis on getting a high score on 3dmark, I stopped during the days of 3dmark 2001 because I found it was not a very reliable way to benchmark a system's or video cards performance. Whenever I read a review on a graphics card, I always skip over the 3dmark section because I wanted to see how it really performs in games and also wanted to see the image quality looks in these games. Right now I own a Radeon 9700 Pro but I'll be picking up a GeforceFX 5900 Ultra as soon as it becomes available (the $399 model, not the $499).

 

[boyRacer]

Originally posted by: AmdInside
I still don't understand why so much emphasis on 3dmark 2003. Personally, I never doubted that NVIDIA and ATI cheat on 3dMark 2003. In fact, I don't even have 3dmark 2003 installed on my PC. While I used to place a lot of emphasis on getting a high score on 3dmark, I stopped during the days of 3dmark 2001 because I found it was not a very reliable way to benchmark a system's or video cards performance. Whenever I read a review on a graphics card, I always skip over the 3dmark section because I wanted to see how it really performs in games and also wanted to see the image quality looks in these games. Right now I own a Radeon 9700 Pro but I'll be picking up a GeforceFX 5900 Ultra as soon as it becomes available (the $399 model, not the $499).

Same here... i just glance over the numbers... but i mainly look for the FPS in whatever games they ran it on.

 

[Pete]

The emphasis here is on driver cheats, not 3DM scores. Let me put it to you in another way: FX cards made big gains in UT2K3 with the latest drivers, too. Are those gains from legit optimizations, or IQ-lowering hacks like we've seen with 3DM? On the topic of UT2K3 gains, it's highly possible they're legitimate driver improvements (Anand didn't spend half his NV35 preview on how cards render 3D for nothing), as it's possible that ATi's GT4 8% increase may also be driver improvements. I'm expecting FutureMark to investigate ATi as thoroughly as they did Nvidia.

 

[Nemesis77]

Originally posted by: AmdInside
I still don't understand why so much emphasis on 3dmark 2003.

Because that benchmark in question was used to prove that NV cheats. Whether you consider the benchmark important or not is irrelevant. What is important is the fact that NV uses dishonest methods in benchmarks.

Whenever I read a review on a graphics card, I always skip over the 3dmark section because I wanted to see how it really performs in games and also wanted to see the image quality looks in these games.

How do you know that those results are any more valid? You can't find out if they are cheating, with 3DMark you can.

 

[waylman]

Originally posted by: Pete
The emphasis here is on driver cheats, not 3DM scores.

Exactly. It seems like the Nvidiots here do not understand this point. I for one, could care less what my 3dmark score is, BUT the fact remains Nvidia CHEATED! Although you may feel (as do I) that 3dmark is not an important benchmark, it is obvious that nvidia was wrong in what they did. Just admit it nvidiots!

 

[gururu]

The new quote from H, which has replaced the other mysteriously missing quote I posted:

According to the PDF on Futuremark's site, the patch causes a 24.1% drop in score for NVIDIA and a 1.9% drop in score for ATi meaning NVIDIA isn't alone in this whole ordeal, other companies have "irregularities" as well. It is good to see Futuremark fix the current situation to remove any unfair advantages, but this goes to show you what we meant in our "Benchmarking Right" article

i have no comment...