Truth or Myth?: Is SYSmark a Reliable Benchmark?

[Phynaz]

AMD joined the Bapco consortium in 2000 because they didn't like the results.

AMD left the consortium over a decade later because they didn't like the results.

In a few years AMD joins the consortium because they don't like the results.

 

[Phynaz]

Did you see second picture i just posted ??? the one with GPU sensitivity ??? R9 290X is 4.8% faster than HD4350 in Media Creation, really ???

Proves nothing.

Edit: except that a system using an R9 290X had 4.8% faster response time than one using an HD4350 in that test.

Anything else is conjecture, not proof.

 

[mysticjbyrd]

AMD joined the Bapco consortium in 2000 because they didn't like the results.

AMD left the consortium over a decade later because they didn't like the results.

In a few years AMD joins the consortium because they don't like the results.

Except when they originally joined they had the better cpus, and yet they still lost this 'benchmark'.

 

[ShintaiDK]

Except when they originally joined they had the better cpus, and yet they still lost this 'benchmark'.

In 2000?

AMD won this benchmark throughout its K8 days.

The only statement AMD made is that they cant compete in CPU performance. And think every single CPU benchmark is bad.

 

[Ansau]

Funny how suddenly the very same applications become relevant, real world applications when GPGPU is used :sneaky:

https://www.youtube.com/watch?v=4CwwQq1ddQ4

And funny how this can also be biased as hell. FX8800p is known for throttling a lot when using cpu+gpu. The cpu being downclocked at only 1.3-1.5GHz and the gpu to 300-700MHz.
http://www.gaminglaptopsjunky.com/hp-envy-15z-review-carrizo-fx-8800p-15w/
http://forum.notebookreview.com/threads/ideapad-y700-15acz-review.783192/
"Theoretically the performance is very good with framerates between 41 and 47 fps (~43 avg) but after 20 minutes i experienced heavy frame drops (last screenshot). Those are caused by the CPU that throttles down to 1,4 GHz for a few seconds and thus makes the game unplayable. ".
"Same behaviour while running Unique Heaven benchmark for half an hour. CPU starts with 3.4 GHz on all threads and throttles down after 15 minutes and stays constantly at 1.4 GHz. The GPU runs with 900-1000 MHz and sometimes drops to ~700.
".

I really doubt it can exceed the i5 when configured at 15W. All these "tests" from AMD can be perfectly cooked to run the APU with an advanced cooling system you wouldn't find in a laptop.

 

[Ma_Deuce]

Can you provide specific details about your system, now I'm curious.
I used Core i3s in the past and even Celeron/Pentium and none of them felt painfully sluggish. Actually 24/7 browsing and light office use is 99% what my Haswell-E delivers. All of them had an SSD though.

The machine I am using daily at the office now is a i3 3220 Dell machine. Nothing fancy. HDD and IGP. Day to day tasks I'm noticing constantly that I have to wait an extra second or two here and there. Not a huge deal, but aggravating when you are working on something time sensitive.

I also use an i5 4670k, i5 3570k, mobile i7, A10-6800K and A8-6600K machine and none of them constantly make me wait like the i3.

Sysmark is in fact a reliable benchmark, it shows you exactly which CPUs can run Sysmark faster. Not unlike any other benchmark. AMDs video is laughable at best, but their point is valid.

 

[Ma_Deuce]

If you're an average customer just buying a laptop for general use though, Sysmark really isn't a great tool to use as a primary determinant of performance. As a single score to determine platform performance, something that also stresses the GPU, the storage subsystem, battery life, etc would be more representative of how they would experience the system in general use.

:thumbsup:

 

[ShintaiDK]

And you excluded it isn't an issue with that PC, rather than the i3?

I never heard anything like that before. Neither have I experienced at home, with Clarkdale, SB and Skylake dualcores with HT.

 

[Ma_Deuce]

And you excluded it isn't an issue with that PC, rather than the i3?

I never heard anything like that before. Neither have I experienced at home, with Clarkdale, SB and Skylake dualcores with HT.

We have several of these machines and they all behave the same. My suspicion is that it's mostly to do with the IGP. Which again, makes the case for the AMD video.

Just my personal experiences and I'm not talking about gaming. /shrug. AMD APU machines have done better in IGP general office builds and if I need anything with more grunt, I go with an i5 and DGPU.

Don't get me wrong, as a company I would be embarrassed to release a video like that lol. Marketing at it's finest and I think it's one of the reasons that AMD is doing so poorly.

 

[mrmt]

Why is/was BAPCo situated at a Intel site??

So,the VP, Sales & Marketing Group & GM, Platform Evaluation & Competitive Assessment was the president of BAPCo,some other bloke in the early 2000s who worked for Intel was a representative of the company and BAPCo offices were located at Intel headquarters.

If that is the case no wonder AMD,Nvidia and VIA all left. Intel is only chip make left at BAPCo.

It's a Consortium, they can use the offices of one of the members without problems, right?

And think about it, AMD wasn't bothered by these facts by the time they were winning on the benchmarks, only when their products became the laughing stock of the market they decided to quit.

The irony is that the failure of the APU strategy, e.g., the iGPU as a added value to consumers was clearly spotted by the benchmark in 2011, with the bigger iGPU/dGPU barely ticking the benchmarks upwards when compared to smaller GPUs. Fast forward A few years AMD was wiped out of the corporate market and their share of business machines cratered. Who was right, AMD marketing team, or BAPCO?

 

[mysticjbyrd]

We have several of these machines and they all behave the same. My suspicion is that it's mostly to do with the IGP. Which again, makes the case for the AMD video.

Just my personal experiences and I'm not talking about gaming. /shrug. AMD APU machines have done better in IGP general office builds and if I need anything with more grunt, I go with an i5 and DGPU.

Don't get me wrong, as a company I would be embarrassed to release a video like that lol. Marketing at it's finest and I think it's one of the reasons that AMD is doing so poorly.

Amd is doing poorly, because they don't have the money to compete with two virtual monopolies at once. Perhaps if their cpu division wasn't dragging them down they might be able to acquire a justifiable market share within the gpu division.

 

[Phynaz]

AMD is doing poorly due to their own screw ups. It's no one else's fault but their own.

 

[mysticjbyrd]

AMD is doing poorly due to their own screw ups. It's no one else's fault but their own.

I am sure that's exactly what the advocates of standard oil said about the competitors as well.

 

[Phynaz]

Unbelievable.

 

[itsmydamnation]

And funny how this can also be biased as hell. FX8800p is known for throttling a lot when using cpu+gpu. The cpu being downclocked at only 1.3-1.5GHz and the gpu to 300-700MHz.
http://www.gaminglaptopsjunky.com/hp-envy-15z-review-carrizo-fx-8800p-15w/
http://forum.notebookreview.com/threads/ideapad-y700-15acz-review.783192/
"Theoretically the performance is very good with framerates between 41 and 47 fps (~43 avg) but after 20 minutes i experienced heavy frame drops (last screenshot). Those are caused by the CPU that throttles down to 1,4 GHz for a few seconds and thus makes the game unplayable. ".
"Same behaviour while running Unique Heaven benchmark for half an hour. CPU starts with 3.4 GHz on all threads and throttles down after 15 minutes and stays constantly at 1.4 GHz. The GPU runs with 900-1000 MHz and sometimes drops to ~700.
".

I really doubt it can exceed the i5 when configured at 15W. All these "tests" from AMD can be perfectly cooked to run the APU with an advanced cooling system you wouldn't find in a laptop.

I dont know about newer Intel CPU's but 15/25 watt haswell is terrible for this, but its actually even worse because it maintains its CPU clock and dropped the GPU clock even further ( the actual thing that is the bottleneck). You can work around it a bit with throttlestop, but its still makes light games like diablo3 unplayable even on low settings.

 

[Azuma Hazuki]

The i3 is badly-positioned, to my mind. It's not enough faster for the people it's targeted at for the price that i can recommend it over the A8, and if they need something faster, I recommend them whichever of i5 or FX-8320E is cheaper in the overall build.

And just below that...again, the A8 isn't expensive enough compared to a Pentium G3xxx that I could recommend against it given the Pentium's lower thread count, even though the Pentium spanks it like a step-headed red child for single thread.

Definitely recommend Celeron G-series over AMD A6 and A4 though; those are just too weak.

Intel needs to either change their HT/thread count SKUs, or make the i3 somewhat cheaper. Here's how I'd do it:

Celeron: 2c/2t, ~$50
Pentium: 2c/4t (we're already seeing these, e.g., Pentium 4405U), ~$80
Core i3: 4c/4t, ~$150
Core i5: 4c/8t, ~$250
Core i7: 6c/12t, ~$350

 

[VirtualLarry]

The i3 is badly-positioned, to my mind.

Intel needs to either change their HT/thread count SKUs, or make the i3 somewhat cheaper. Here's how I'd do it:

Celeron: 2c/2t, ~$50
Pentium: 2c/4t (we're already seeing these, e.g., Pentium 4405U), ~$80
Core i3: 4c/4t, ~$150
Core i5: 4c/8t, ~$250
Core i7: 6c/12t, ~$350

I like that idea. Especially the idea of a Pentium with HT. It's due time, really. The i3 can retain the added instructions like AVX/AVX2. But give Pentium HT, and Intel has already, in SKL, given them AES-NI and VT-d.

 

[mysticjbyrd]

Unbelievable.

Everyone makes mistakes, but if Amd was getting a fair market share for their products, they wouldn't ended up in this situation. We would have a lot more competition, and technology would have advanced further.

Competition is good for consumers, bad for monopolies.

 

[mrmt]

Everyone makes mistakes, but if Amd was getting a fair market share for their products, they wouldn't ended up in this situation. We would have a lot more competition, and technology would have advanced further.

Competition is good for consumers, bad for monopolies.

AMD isn't being prevented from selling their products. In fact, there's a lot of AMD products on the shelves but nobody buys them. AMD could get a fair share of sales if they designed products people actually want to buy, and not pipe dreams.

 

[ShintaiDK]

We have several of these machines and they all behave the same. My suspicion is that it's mostly to do with the IGP. Which again, makes the case for the AMD video.

Just my personal experiences and I'm not talking about gaming. /shrug. AMD APU machines have done better in IGP general office builds and if I need anything with more grunt, I go with an i5 and DGPU.

Don't get me wrong, as a company I would be embarrassed to release a video like that lol. Marketing at it's finest and I think it's one of the reasons that AMD is doing so poorly.

So you try to advocate for weak CPUs in the benefit of IGPs. Yeah....

I´m sorry, I need some hard facts on this one. Else I have to say that I dont believe you the slightest.

 

[ShintaiDK]

AMD is doing poorly due to their own screw ups. It's no one else's fault but their own.

Exactly. Anyone thinking otherwise is in denial.

 

[ShintaiDK]

Everyone makes mistakes, but if Amd was getting a fair market share for their products, they wouldn't ended up in this situation. We would have a lot more competition, and technology would have advanced further.

Competition is good for consumers, bad for monopolies.

AMD is getting their fair share. But AMD dont have compelling products to sell. Result, low sales that keep declining.

 

[mysticjbyrd]

AMD isn't being prevented from selling their products. In fact, there's a lot of AMD products on the shelves but nobody buys them. AMD could get a fair share of sales if they designed products people actually want to buy, and not pipe dreams.

There is a lot more to this than the best product, especially in the past. That's not how capitalism works in the real world, especially not in the US.

AMD is getting their fair share. But AMD dont have compelling products to sell. Result, low sales that keep declining.

You could argue that for their current cpus, but not in the past. It's also not true of their gpu market.

 

[Techhog]

The only statement AMD made is that they cant compete in CPU performance. And think every single CPU benchmark is bad.

You have more examples?

 

[Ma_Deuce]

So you try to advocate for weak CPUs in the benefit of IGPs. Yeah....

I´m sorry, I need some hard facts on this one. Else I have to say that I dont believe you the slightest.

Oh noes! Someone on the interwebs doesn't believe me. It's OK, you aren't required to.

Either way, the point that AMD was getting at in their video is valid. Sysmark is not a reliable benchmark for anything other than seeing which machine can run sysmark the fastest.