FutureMark & Nvidia joint statement on 3DMark03; FutureMark tucks its tail between its legs.

[Dean]

This is nothing but a complete mess now. It goes to show what can happen when online reviewers(the larger established ones) are afraid of exposing one of the big boys(their meal ticket). The entire hardware community is going to split over this issue and nobody will win in the end.

 

[WarCon]

I sure hope some company rushes into fill this void (I wish fabr-rausch was more business oriented) and give us a benchmark that doesn't require any figiting from the outside (no manufacturer involvement whatsoever). Where the company would make a video benchmark based on the current video standards and rate each card based on DirectX 7, 8, and 9 performance separately and on opengl separately. Also introduce a random nature to the scene flow, so that we can put a stop to static path cheating. Have the benchmark analysing the output for quality (like how effective is one cards 4x anisitropy in smoothing a diagonal line/plane), have it test shaders. I personally think that some ansitropy and antialiasing is not only inefficient but almost ineffective too and I would like to see individual elements of modern cards get tested specifically.

And I know that I don't play the benchmarking tool, but if it is a tool that I can trust (Futuremark just proved itself to be a whore or a wuss so trusting them would be foolish) then I can tell which $100 or $200 or even $300 card I can place my faith in while trying to decide what to purchase. I don't have the money to buy and try as many seem to and when I consider spending as much on my video card as I do on my motherboard, processor and memory combined then I would really like to be able to say I picked the best and am not just paying for the name.

 

[NFS4]

DAMN!!! That Rage3D thread is HARDCORE:Q

 

[Jeff7181]

I posted in that discussion, but I'll also post it here for those too lazy to read it all...

"I said it from the first time I saw a mention of this on the boards... optimization of hardware/hardware drivers for individual software programs is a good thing, and is not cheating.
People claim they sacrificed image quality, well yeah, that's a bad thing, but if they can create optimizations for specific games that increase performance and image quality is the same or better, go for it... that's what I'm paying for when I pay $500 for a video card... I want some extensive R&D to go into new products."

Exactly. This fiasco just tells me that without these special FX drivers, the 5900 Ultra can't compete with ATi's offerings. R3XX core pwns.

So what you're saying is... nVidia isn't allowed to create drivers for their hardware? Who gives a flying cow turd WHAT kind of drivers it needs... the fact is, it does what it's supposed to do.

Man... people need to stop with all the gossip and soap opera crap... a performance increase is a good thing for us, the consumer. If nVidia wants to write tons of lines of code to optimize performance, that's fine, that's one way of doing it... if ATI wants to create a monster of a GPU to optimize performance, that's fine too, that's another way of doing it. Stop with all the scandals... do you see how stupid the argument is? People are complaining that nVidia increased performance, but they didn't do it the same way ATI does it, so they're a horrible horrible company cause they didn't play by the rules. Who "forkin" cares? If there was no competition things would be a lot darker than they are now.

 

[Oalex]

Originally posted by: Jeff7181
I posted in that discussion, but I'll also post it here for those too lazy to read it all...

"I said it from the first time I saw a mention of this on the boards... optimization of hardware/hardware drivers for individual software programs is a good thing, and is not cheating.
People claim they sacrificed image quality, well yeah, that's a bad thing, but if they can create optimizations for specific games that increase performance and image quality is the same or better, go for it... that's what I'm paying for when I pay $500 for a video card... I want some extensive R&D to go into new products."

Exactly. This fiasco just tells me that without these special FX drivers, the 5900 Ultra can't compete with ATi's offerings. R3XX core pwns.

So what you're saying is... nVidia isn't allowed to create drivers for their hardware? Who gives a flying cow turd WHAT kind of drivers it needs... the fact is, it does what it's supposed to do.

Man... people need to stop with all the gossip and soap opera crap... a performance increase is a good thing for us, the consumer. If nVidia wants to write tons of lines of code to optimize performance, that's fine, that's one way of doing it... if ATI wants to create a monster of a GPU to optimize performance, that's fine too, that's another way of doing it. Stop with all the scandals... do you see how stupid the argument is? People are complaining that nVidia increased performance, but they didn't do it the same way ATI does it, so they're a horrible horrible company cause they didn't play by the rules. Who "forkin" cares? If there was no competition things would be a lot darker than they are now.

You can't seriously be this blind!!!! edit: (or biased)

Static Planes are not optimization, its fake enviromant = bad/wrong image = cheats.
These benchies are suppose to simulate workload and calculate performance base on that. Reducing the work load is not an optimisation, because you don't get better performance, but a fake performant (i.e. the effective work at question is OMITED not optimezed).

I guess this is too high for you to comprehend thogh, for not understanding on you own, what ever one else sees clearly.

 

[Jeff7181]

Originally posted by: Oalex

Originally posted by: Jeff7181
I posted in that discussion, but I'll also post it here for those too lazy to read it all...

"I said it from the first time I saw a mention of this on the boards... optimization of hardware/hardware drivers for individual software programs is a good thing, and is not cheating.
People claim they sacrificed image quality, well yeah, that's a bad thing, but if they can create optimizations for specific games that increase performance and image quality is the same or better, go for it... that's what I'm paying for when I pay $500 for a video card... I want some extensive R&D to go into new products."

Exactly. This fiasco just tells me that without these special FX drivers, the 5900 Ultra can't compete with ATi's offerings. R3XX core pwns.

So what you're saying is... nVidia isn't allowed to create drivers for their hardware? Who gives a flying cow turd WHAT kind of drivers it needs... the fact is, it does what it's supposed to do.

Man... people need to stop with all the gossip and soap opera crap... a performance increase is a good thing for us, the consumer. If nVidia wants to write tons of lines of code to optimize performance, that's fine, that's one way of doing it... if ATI wants to create a monster of a GPU to optimize performance, that's fine too, that's another way of doing it. Stop with all the scandals... do you see how stupid the argument is? People are complaining that nVidia increased performance, but they didn't do it the same way ATI does it, so they're a horrible horrible company cause they didn't play by the rules. Who "forkin" cares? If there was no competition things would be a lot darker than they are now.

You can't seriously be this blind!!!!

Static Planes are not optimization, its fake enviromant = bad/wrong image = cheats.

I don't care if they stick bubble gum on the bottom of their cards if it increases performance. Decreasing image quality is one thing... creating application specific drivers is another.

 

[308nato]

Originally posted by: GTaudiophile

Originally posted by: ElFenix
sounds like nvidia paid them off because inserting clipping planes isn't just using a more efficient path

Of course, everyone at Rage3D has already come to that conclusion.

Well....if the members of rage3d believe it it must be so. They are such a mature and insightful group.

 

[Oalex]

Originally posted by: Jeff7181

Originally posted by: Oalex

Originally posted by: Jeff7181

I don't care if they stick bubble gum on the bottom of their cards if it increases performance. Decreasing image quality is one thing... creating application specific drivers is another.

But don't you see it not optimization:

These benchies are suppose to simulate workload and calculate performance base on that. Reducing the work load is not an optimisation, because you don't get better performance, but a fake performance (i.e. the effective work at question is OMITED not optimezed).

While painting your house, if you omite to paint the kitchen, then you are not increasing performance, but reduceing time it takes you to NOT complete the job. In other words you cant say that you painted the house, just painted a few rooms. period.

I guess this is too high for you to comprehend thogh, for not understanding on you own, what ever one else sees clearly.

 

[Jeff7181]

Originally posted by: Oalex

Originally posted by: Jeff7181

Originally posted by: Oalex

Originally posted by: Jeff7181

I don't care if they stick bubble gum on the bottom of their cards if it increases performance. Decreasing image quality is one thing... creating application specific drivers is another.

But don't you see it not optimization:

These benchies are suppose to simulate workload and calculate performance base on that. Reducing the work load is not an optimisation, because you don't get better performance, but a fake performance (i.e. the effective work at question is OMITED not optimezed).

While painting your house, if you omite to paint the kitchen, then you are not increasing performance, but reduceing time it takes you to NOT complete the job. In other words you cant say that you painted the house, just painted a few rooms. period.

I guess this is too high for you to comprehend thogh, for not understanding on you own, what ever one else sees clearly.

I think a better analogy would be that instead of putting 2 coats of paint, I buy paint that's guaranteed to cover with just one coat. nVidia isn't "skipping" anything... they're finding a better, more efficient way of doing it.

 

[Oalex]

I think a better analogy would be that instead of putting 2 coats of paint, I buy paint that's guaranteed to cover with just one coat. nVidia isn't "skipping" anything... they're finding a better, more efficient way of doing it.

EER...wrong: making a plane that is supposed to be moving static is not a second coat of paint, its like using wall-paper instead of paint.
Nvidia could then say that they wall-paperd the kithen in 5 sec. (WallpaperMak03 score 50,000), but they sould not claim that they painted the kitchen in 5 sec., cause the 3dmark03 requiereds them to USE PAINT (i.e. flying planes), for comparison purposes.

 

[Jeff7181]

Originally posted by: Oalex

I think a better analogy would be that instead of putting 2 coats of paint, I buy paint that's guaranteed to cover with just one coat. nVidia isn't "skipping" anything... they're finding a better, more efficient way of doing it.

EER...wrong: making a plane that is supposed to be moving static is not a second coat of paint, its like using wall paper instead of paint.
Nvidia could then say that they walpaperd the kithen in 5 sec. (WallpaperMak03 score 50,000), but they sould not claim that they painted the kitchen in 5 sec., cause the 3dmark03 requiereds them to USE PAINT, that is for comparison purposes.

You're right, I'm wrong... there, happy now?

 

[Oalex]

You're right, I'm wrong... there, happy now?

not sure....abotu the happyness) more supprised (almost dissaponited) than happy a lot off treads I read here show that a lot of ppl are biggots.

You are not one of them! they don't give up so easily, even with reason.
on the other hand I AM starting to feel happy now, not in a bad way. You Mr. are ok.

 

[TROGDORdBURNINATOR]

I realize that people are going to take WHATEVER I say to task just because of who I am regardless of what I say. Heck, I've even praised Nvidia for one or two things and been attacked for it. Anyway, I think that Nvidia does have a point with what they're saying. As we can clearly see from Doom 3 which has its own NV3x rendering path, in the real world game developers do optomize their games for certain hardware. They didn't optomize doom 3 for the R3xx because it wouldn't have been as effective. As a result, the Nvidia cards run the game faster. Is this reflected in 3dmark though? No...

because it doesn't allow optomization. That's its worst flaw. The Nv3x core is one that requires optomization to be effective (much like the pentium 4.) r3xx is a lot like an Athlon. Works great with standard code. 3d mark not taking into account the uniqueness of the NV3x core and its need for optomization which does happen in games is a lot like PCmark or Sandra not using SSE2 in its benchmarks because it's a P4 only optomization (when the benchmark was written.) That's ridiculous.

Nvidia got upset when futuremark didn't agree with this and built their bench without manufacturer optomization. Outdated, useless bench is what it became.

However, what Nvidia is doing now is wrong. They're not optomizing, they're cheating. They're trying to discredit the benchmark any way that they can because they feel that they've been wronged. I think that they have actually. But I don't think it's right to destroy a benchmark because you don't agree with its testing methodology.

I do agree with Nvidia that 3dmark is obsolete and makes no sense. The benchmark should have optomized rendering paths for different hardware. It's only fair because some hardware needs it more than others (p4 vs athlon xp) and real world games have it which is what this bench is supposed to reflect.

In short:
Futuremark: bad
Nvidia" bad

 

[Harabecw]

Why won't the R3xx benefit as much from optimizing? I heard that being said quite a while ago...I think in one of Carmack's .plans.
Is there a place to read about how FX and R3xx cards are optimized?
Or is that too technical?

 

[Jeff7181]

Originally posted by: TROGDORdBURNINATOR
The benchmark should have optomized rendering paths for different hardware. It's only fair because some hardware needs it more than others (p4 vs athlon xp) and real world games have it which is what this bench is supposed to reflect.

Games are designed to use these features... 3DMark is designed to measure general gaming performance... wouldn't it make sense that if features are used in real games, that a program which tests gaming performance should reflect the performance increase that these features provide?

Maybe to make everybody happy, futuremark should get permission from these gaming companies to put REAL games in them. Like a Quake 3 demo, and a UT2k3 demo. THEN... use some sort of key logger to program in the exact same movement and views so people who use the benchmark can't cheat and just stare at a wall and say "I get 400 frames per second in UT2k3"

*EDIT* I realize they're saying 3DMark is supposed to test unoptimized hardware... but what good is that if all games have optimizations to make better use of the hardware?

 

[Gstanfor]

I can't believe people are still attacking nVidia over the extra clipping planes.

These are some of the optimizations they wanted Futuremark to include in 3DMark 2003 to begin with and Futuremark would not.

The extra clipping planes if included in the actual program would otpmize performance for all 3d card makers.

Nothing is being clipped that can actually be seen. The 3DMark2003 demo is a fixed viewpoint demo with no opportunity for the enduser to vary his point of view into the scene. Thus clipping what is outside of that viewpoint is a legitimate optimization.

Unless someone can come up with a freely downloadable, legal version of 3DMark2003 that allows users to freely look around the scene(s) at any point in the benchmark, you have nothing to complain about.

 

[MikeMike]

how fast history is forgotten.


i seem to remember that ATI was once caught cheating on some quake benchmarks. all's fair in love and war.

MIKE

 

[DeathByDuke]

damn, just what we need, a Microsoft in the Video market....

 

[GTaudiophile]

Originally posted by: nourdmrolNMT1
how fast history is forgotten.


i seem to remember that ATI was once caught cheating on some quake benchmarks. all's fair in love and war.

MIKE

Yep, and ATi got grilled. nVidia is getting away with it like a fart in the wind. Talk about double standards, no?

 

[MikeMike]

Originally posted by: GTaudiophile
You all should read this very heated thread over at Rage3D.com! The Rage3D members are having a field day with Kyle from HardOCP (frgmstr).

CATALYST maker (a software/driver developer for ATi) had this to say:


OK time to get in the fun.

ITS CM TIME!!!!

First of all as a favor to me, leave Kyle and Brent out of this. This has nothing to do with them. Period

Secondly I am a little upset tonight so I wont say much until tomorrow.

Third I guarantee you that I will ask for an investigation for optimized drivers tomorrow such that has never happened in ATI's history. I am prepared to put a hold on all new features I have in the pipeline so our top engineers can see how much we can optmize by not rendering the whole scene. I am guessing we can gain 25% at this point.

Fourth I am not commiting to do these optimizations ever in a released driver but I think its time for apples to apples comparison.

Fifth I am sorry to hear you (the end consumers) so dissapointed in the state of the industry. I feel for you.

Sixth, you all have my personal guarantee that if you continue to support ATI the way you have so far, I will always be here to help out and be one of the boyz on the forums. (I hope that means something to at least some of you)]

Have a good night everyone and lets talk more tomorrow
Terry

what exactly does he mean by that? didnt they do optimizations for the quake benchmarks?

and gt, nvidia isnt getting away with it, they are getting grilled like its the first time anyone has ever cheated on anything.

raise your hand if you ever cheated on a test. (raises hand)

thats what i thought, 100% of the people raised their hands.

MIKE

 

[Gstanfor]

IMO Microsoft is the guilty party behind all the nVidia controversy of the last few months.

Every thing was fine until nVidia came out on top of the X-Box abrtitration dispute. Right about that time, the DirectX9 specification on minimum allowed precisions changed - in ATi's favor (NV3x was designed around being able to use multiple precisions, depending on the needs of the rendering task). I see that as Microsofts' revenge for losing the arbitration (and having to pay nVidia more for each x-box chipset).

It isn't a unique trend either. nVidia's rapid and strong growth seems to have ruffled a few established feathers in the industry. Intel will not grant nVidia a competitive license to use nForce and nForce2 for P3 & P4 (looking to protect their own vested interest in the motheboard chipset market ~60% of total market).

Futuremark and ATi jumped on a golden opportunity to make nVidia look inferior by comparison.

It will be very interesting indeed to see how upcoming game titles are coded. I am willing to bet that more games will adopt the nVidia/original DX9 route of multi precision rendering than the ATi/new DX9 method (full precision only, with hints for lower levels if supported).

 

[NFS4]

Originally posted by: GTaudiophile

Originally posted by: nourdmrolNMT1
how fast history is forgotten.


i seem to remember that ATI was once caught cheating on some quake benchmarks. all's fair in love and war.

MIKE

Yep, and ATi got grilled. nVidia is getting away with it like a fart in the wind. Talk about double standards, no?

Yup, NVIDIA must have pulled out the Rambus lawyers

 

[Harabecw]

didn't RAMBUS come out as the better company out of the whole deal?

 

[Adul]

http://www.hardocp.com/article.html?art=NDgy

Kyle has yet another thing or three to say about this.

 

[WarCon]

I wish I knew how deep Kyle's hand goes into NVidia's pocket. I like the reference to a "new" benchmarking utility that will be coming out later this year. I am sure it will show NVidia to be head and shoulders above the competition and will be the "next" big thing (if NVidia dumps enough money into its advertizing).

I personally just despise this whole charade.