Response from FS to HoCP "real world" tests.

[thilanliyan]

click

Frankly, I don't like the "real world" testing done by HOCP. In vid card tests they basically put the settings so that the cards come up about even in fps and then pick a winner...it's a bit difficult to discern which features have a larger impact though since they're not trying it on each card. Benchmarking 2 differect sets of hardware should mean they keep the application settings the same and then pick the winner, at least in my opinion.

The Conroe test was laughable...we already know how a 7900GTX performs, we didn't need a "cpu test" to show it.

HardOCP rebuttal:
click

way to take the high road...

 

[theprodigalrebel]

I read the Firing Squad article and then Kyle's rebuttal on HardOCP.

Firing Squad made some very, very valid points and went about their business in a tactful manner. They were pissed at HardOCP declaring itself the King of Reviewers and calling the rest of the Internet trash, BS and garbage.

How does HardOCP react? BY NAME CALLING! I can do it too: Kyle, get raped by a rabid goat. You are (gasp!) not the center of the universe. And you totally failed to answer the questions thrown by Firing Squad: Why wasn't multi-GPU considered? What about the mention of the playnettimedemo which includes AI & Physics in a timedemo? The difficulty experienced in creating repeatable tests for games like Oblivion? Testing at multiple resolutions?

Firing Squad didn't say they were 'better' than the rest: they said both methods of testing have their own unique pros/cons and consumers do benefit from seeing varied approaches. They were offended by HardOCP's attitude: "You are with us or with them - and they are all stupid"

HardOCP could only indulge in some mud-slinging and name-calling instead of answering questions. HardOCP's credibilty just went that way---->:thumbsdown:

 

[imported_wicka]

Frankly, I thought we've all known for years that H was full of crap. I certainly have. No one should see it as a shock that:
1. They try to claim they are superior to all other review sites.
2. Their response contains no technical value, only canned responses and name calling.

 

[imported_heyguy]

I wonder when Kyle is going to get tired of saying "canned" and "real world." I hope it's soon. Or maybe I'll just stop visiting there.

 

[akshayt]

You can't put down harcop test just becuase they used a 7900GTX, that too I think was overclocked, not sure.
There is not so much difference in its performance than with 1900XT and the difference is certainly not that much that it would remove the GPU bottleneck. In fact there are games where a 1900XTX will loose badly.

The test was fully fine. If you have a low end card then c2d increases performance, if you have cf/sli high end then also it increases performance, but with single high end card in the games that were tested it doesn't nicrease performance except in Oblivion.

 

[Zugzwang152]

I'll be sticking to AT and sometimes Tom's reviews.

 

[Effect]

Originally posted by: akshayt
You can't put down harcop test just becuase they used a 7900GTX, that too I think was overclocked, not sure.
There is not so much difference in its performance than with 1900XT and the difference is certainly not that much that it would remove the GPU bottleneck. In fact there are games where a 1900XTX will loose badly.

The test was fully fine. If you have a low end card then c2d increases performance, if you have cf/sli high end then also it increases performance, but with single high end card in the games that were tested it doesn't nicrease performance except in Oblivion.

Even my cat could have predicted the same results as ardOCP, and it doesn't claim to be a reviewer. Did anyone NOT know that (high res.) graphics depend on the GraphicsPU rather than CPU (provided the CPU isn't bottlenecking)?

 

[hans007]

hardocp is the worst review site. they also are the most easily bought.

they are exactly the same about anything BFG makes. there was a huge controversy when they kept saying the BFG nforce4 boards were so great, etc etc, when they were in fact all made by chaintech (vnf4 ultra models) . same with all the evga motherboards that are actually made by jetway.

the worst was that, they would always look down on the chaintech boards, sine they were very cheap like $80, but when bfg sticks itin a box and prices it for twice as much, all of a sudden there is "r&d and tech support and all this crap" that makes it a fantastic board.

 

[thilanliyan]

Originally posted by: akshayt
You can't put down harcop test just becuase they used a 7900GTX, that too I think was overclocked, not sure.
There is not so much difference in its performance than with 1900XT and the difference is certainly not that much that it would remove the GPU bottleneck. In fact there are games where a 1900XTX will loose badly.

The test was fully fine. If you have a low end card then c2d increases performance, if you have cf/sli high end then also it increases performance, but with single high end card in the games that were tested it doesn't nicrease performance except in Oblivion.

Please, the post wasn't about that they used a 7900 instead of an X1900, it was that they tested at high res meaning the GPU was the bottleneck in their "CPU" test. What's the point of doing a "CPU" test where the GPU is the bottleneck??

 

[classy]

I like what hardocp did and I hope more reviewers start doing the same. They setup systems and benchmarked them in the way WE ACTUALLY USE THEM. If there is some fallacy in that I can't find it. I have been saying for years the way sites test hardware is flawed. Who cares in you get 1000 fps at 640x480? I don't play at that resolution. I also play with sound on as well and the last time I checked I don't turn off my anti-virus when I am gaming either. Its about damn time somebody started showing results of what you will actually see. The forums are full of threads "I was expecting more" type quotes because they read a review that did not shed light on they way the system would work when used by the person in everyday life. They told the truth simple as that.

And as far as good sites, both of those sites are in my top 10, I like them both a lot.

 

[thilanliyan]

Originally posted by: classy
I like what hardocp did and I hope more reviewers start doing the same. They setup systems and benchmarked them in the way WE ACTUALLY USE THEM. If there is some fallacy in that I can't find it. I have been saying for years the way sites test hardware is flawed. Who cares in you get 1000 fps at 640x480? I don't play at that resolution. I also play with sound on as well and the last time I checked I don't turn off my anti-virus when I am gaming either. Its about damn time somebody started showing results of what you will actually see. The forums are full of threads "I was expecting more" type quotes because they read a review that did not shed light on they way the system would work when used by the person in everyday life. They told the truth simple as that.

And as far as good sites, both of those sites are in my top 10, I like them both a lot.

What you're talking is about benchmarking SYSTEMS...not individual components, and so is even less useful since we don't all have the same combination of components that make up our systems.

 

[classy]

Originally posted by: thilan29

Originally posted by: classy
I like what hardocp did and I hope more reviewers start doing the same. They setup systems and benchmarked them in the way WE ACTUALLY USE THEM. If there is some fallacy in that I can't find it. I have been saying for years the way sites test hardware is flawed. Who cares in you get 1000 fps at 640x480? I don't play at that resolution. I also play with sound on as well and the last time I checked I don't turn off my anti-virus when I am gaming either. Its about damn time somebody started showing results of what you will actually see. The forums are full of threads "I was expecting more" type quotes because they read a review that did not shed light on they way the system would work when used by the person in everyday life. They told the truth simple as that.

And as far as good sites, both of those sites are in my top 10, I like them both a lot.

What you're talking is about benchmarking SYSTEMS...not individual components, and so is even less useful since we don't all have the same combination of components that make up our systems.

What a weasle response. When I buy any component, I am buying based on what impact it will have on my system. If I used your thought process, 99% of all people shouldn't read anything from a hardware site then, because 99% of the people can't afford the system configurations used to test hardware with at most of the sites anyway.

Here is the reality in the whole Conroe aftermath, if your building for gaming the video card is the most important piece and any of these processors on the market can easily provide plenty of power to have top notch gaming. Conroe is the best cpu for both gaming and its encoding and rendering power is incredible. So if do a lot of desktop stuff as well then go Conroe. Conroe is the better cpu, but for right now AMD has the better platform. AM2 is boards are cheaper, fast, and solid. Conroe doesn't have that yet. There is really only one board for Conroe and that will set you back $250.

 

[Noubourne]

Well I never read either of these sites for reviews.

However, the "playable settings" comparison junk makes the H reviews completely worthless imo, and I stopped reading their site altogether because of it. Other reviews of graphics cards clearly show where one company exceeds and another is lagging (D3D vs. OpenGL), and the H reviews take that right out so you have to sit there and compare resolutions and graphics settings, instead of just being able to look at it and say which card does better.

Typically new hardware is within a few percent of each other performance wise. You can't see that easily when you have to pore over which resolution some guy considers "playable". It also make it hard to see when a new generation of hardware is MORE than a few percent faster, because again, the lame-ass subjective "playable" resolution BS strikes again.

In the case of C2D, I like the FS method better. But truth be told, I only looked at it to compare it to H methods. I don't read FS.

 

[R3MF]

the was an eye-opener in so far as the value in spending a fortune on a CPU if you game at high-res.

i game at 1920x1200 so i doubt it makes more than a 2% increase in performance to go from a £173 X2 4600 to a £700 X6900 EE.

i may be persuaded to go for a C2D 6600, but mainly because the mATX motherboards are available with a 16x/PCI/PCI/1x expansion configuration, whereas AM2 mATX boards have a awkward 1x/16x/PCI/PCI configuration.

 

[Faikius]

I personally didn't have a problem with the point HOCP was trying to make. However I don't think they should've tried to make that point in the Core2Duo review.

A much more professional way to have gone about it would have been to have two seperate articles:

1) Core 2 Duo review

2) The Effect of CPU Speed / Power on High-Res Gaming

But then again, this approach wouldn't have gotten nearly as much attention for them

 

[tuteja1986]

-remeber when kyle killed its famous forum and opened it back but you had to pay to post
-or the time he banned everyone that went against Nvidia in AEG crysis
-or the time where he abused this poor website that revealed Nvidia was cheating with its driver... he said that the website was telling a lie and only reason it said that because nvidia wouldn't let them benchmark doom 3

Err , Kyle is an real asshole and he is a total Nvidia's Bitch.

 

[thilanliyan]

Originally posted by: classy
What a weasle response. When I buy any component, I am buying based on what impact it will have on my system. If I used your thought process, 99% of all people shouldn't read anything from a hardware site then, because 99% of the people can't afford the system configurations used to test hardware with at most of the sites anyway.

I think you misread what I said. 99% of the review sites isolate and test components so of course you can use those reviews to pick a component. If a CPU test involves a GPU bottleneck(what HOCP did), how is that isolating and testing the CPU???

Again, if you don't isolate the components, there are too many variables introduced into the test.

There is definitely a place for HOCPs tests, but other sites tests are just as valid, unlike what Kyle is claiming.

 

[BD2003]

While HardOCP has a point in saying that the benchmarks most if not all other sites show somewhat exaggerate what gains we would get from a product, this is only a valid point for a VERY SMALL section of the population that has neither a brain to do a simple self-analysis, and the exact same configuration that HOCP is using. The real world has an infinite different configs, so how can they say that choose one modest one is more real world than any other?

The simple fact is, when you take firingsquads article in a vacuum, it may appear to be missing something. But taking it in context of the rest of the site, where not only hardware, but games are benchmarked, you can see exactly how certain components affect the games.

In essence, the FS numbers are artificial. The OCP numbers are PURELY subjective. Ideally, an article would contain both. Neither is completely wrong, but if there was only going to be one type of benchmarking, FS non-bottlenecked, objective benchmarks by a mile.

 

[imported_wicka]

Basically, Kyle shows one good point (that CPU has very little effect on high-end gaming). The sad thing is: we all know this. H doesn't appeal to us. H appeals to idiots who think they're cool because they read it every day (and now I'm NOT refering to people above who said they like H). As long as they get their advertising revenue, they couldn't care less. And by bashing every other review site, they've taken their whole audience of n00bs and made them feel "1337," and thus far more loyal.

 

[tuteja1986]

Well i have read kyle article for years and i know he is going eat alive Firing SQUAD and sad bit is Firing Squad is one of best Real enthusiastic GPU review website. They are the rare website that do really high res testing with max AA/AF setting.

XBIT LABS , FIRING SQUAD , RAGE 3D are my fav GPU review website.

 

[BD2003]

Originally posted by: wicka
Basically, Kyle shows one good point (that CPU has very little effect on high-end gaming). The sad thing is: we all know this. H doesn't appeal to us. H appeals to idiots who think they're cool because they read it every day (and now I'm NOT refering to people above who said they like H). As long as they get their advertising revenue, they couldn't care less. And by bashing every other review site, they've taken their whole audience of n00bs and made them feel "1337," and thus far more loyal.

Exactly.

 

[dexvx]

LoL

Where were all the AMD fanbois when the Pentium-D 920 could match an FX-60 due to the GPU bottleneck? All you fanbois saying that the P-D 920 = FX-60?

Moreover, HardOCP benchmarks are fuxxed up. The AMD Test systems used a "lighter"configuration (in the config Kyle noted that the AMD system had dynamic shadows turned off). So much for that.

 

[nyker96]

I think real world does have its values, but bottlenecking the system with a GPU isn't benchmarking. But I also have to say I do agree with HCop, we do need some real world benches to put some sense into this "20%" faster, 50% better cliams. As far as I know, at least you need to say 50% faster if xxx.

In real world all that 20%//50% might or might not manifest, so reviewers should make it clear that you need say a X1900+ g-card to see a 20%+ different at xxx resolution.

 

[theprodigalrebel]

Firing Squad didn't blast 's testing methodology: they said everyone stands to benefit from different perspectives and a wider range of testing. The reason they posted their Rant was because seems to think they are the best and everybody else can DIAF. 's rebuttal looked like it was written by a drunk 12-year-old and it completely failed to address the points in FS's rant.

needs to learn a few things from Anandtech, Firing Squad, X-bit Labs and PCSTATS on professionalism. They are more than welcome to stick to their methodologies. At least incidents like these remind us of the credibility of various tech-review sites.

 

[Suspicious-Teach8788]

Originally posted by: nyker96
I think real world does have its values, but bottlenecking the system with a GPU isn't benchmarking. But I also have to say I do agree with HCop, we do need some real world benches to put some sense into this "20%" faster, 50% better cliams. As far as I know, at least you need to say 50% faster if xxx.

In real world all that 20%//50% might or might not manifest, so reviewers should make it clear that you need say a X1900+ g-card to see a 20%+ different at xxx resolution.

Bottlenecking a system with a GPU when there are ways to not bottleneck it is just WRONG.

If you're buying a 7900GT, I don't think it's that much of a stretch to get SLI at that point. You might as well bench wtih a freaking 7600 or 6600GT if you're going to GPU limit it (basically for mid-range buyers).

Real world has benefits, but honestly, if we didnt have fanboiism, I swear we wouldn't really be seeing so many real world benchmarks.

Originally posted by: dexvx
LoL

Where were all the AMD fanbois when the Pentium-D 920 could match an FX-60 due to the GPU bottleneck? All you fanbois saying that the P-D 920 = FX-60?

Moreover, HardOCP benchmarks are fuxxed up. The AMD Test systems used a "lighter"configuration (in the config Kyle noted that the AMD system had dynamic shadows turned off). So much for that.

Like I said, if we flipped it the way around and Opterons really killed Conroes the same way Conroes kill Opterons today, what would you say if we showed Opteron benchmarks being GPU limited? You would hear millions scream. Just because Intel sucked yesterday doesn't mean you give them this huge bottleneck so their victory is reduced simply to appease AMD fanboys.