ARM vs X86 power efficiency showdown

[ncalipari]

Lately there has been a discussion about if x86 can or cannot be more power efficient than ARM.

My null hypothesis is:

ARM is more efficient of x86 in the 1-10W power envelope, due to the lack of a power expensive decode step and missing legacy burden of an ISA not any more adapat to the modern world. This holds especially for the integer operations.


The goal is to measure the efficiency of a FULL system, so we will need to take account of radio, heat, memory, GPU and all the other specifications that are intimately related to the CPU architecture (so the display doesn't matter).

The efficiency should be computed at power socket level, over a series of benchmarks. My suggestion is to use Java Integers test in a Linux environment. Android would skew this comparison, has it has a software that is GREATLY optimized for ARM.

It would be better if the test could be done with the machines attached to an UPS in a 42 C (110 F) room.


The candidate platforms should be:

ARM:

- Samsung Origen

- ST Ericson Snowball

- FreeScale i.MX54

TI Omap is not available to the public in a power efficient envelope, you need to purchase huge volumes to access particular configurations, so for now I would leave it as it is.

x86

- AMD Brazos

- Intel Atom


We can only choose:

- Platforms that are commercially available at the moment of test

- Systems with a total peak power consumption of 25 W or less (radio included).


I have a Samsung Origen at home. We need to find as much people as possible with the other platforms.

We should also define a set of java benchmarks.

Anyone has any suggestion on the methodology, or feels willing to write a MethDoc or a Java Bench?

 

[OVerLoRDI]

Ambitious goal and I'd love to see your results and the benchmarks you devise.

I think you may find that x86 right now cannot get down to the power levels ARM can. Intel is working hard to change this.
http://www.technologyreview.com/computing/39378/page1/

The question is, can process tech win the day? Or will the more efficient architecture win? The next 5 years are going to be very interesting.

 

[Mr. Pedantic]

How is that a null hypothesis? Wouldn't it be thinking that there are no significant differences in power efficiency between the ARM and x86 architectures?

The efficiency should be computed at power socket level, over a series of benchmarks. My suggestion is to use Java Integers test in a Linux environment. Android would skew this comparison, has it has a software that is GREATLY optimized for ARM.

Just before this, you wrote that the purpose was to test the efficiency of a full system. How is Android not part of the full system?

 

[dealcorn]

The fact that this thread is in a public forum enhances the site's credibility which is a valued asset. Still, you made a boo boo in stating a null hypotothis. It might be read that you are either looking for tests that will confirm your desired result or pandering to an interest group. I suspect neither was your intent. Think Ceasur's wife.

Regarding Android, it was highly optimized for ARM and Intel either was or was not partially or fully successful in performing its own optimizations. From a performance/efficiency perspective, consumers care the results, not history. From a skill set perspective, this is not a level playing field. Chat it up however you want, but Android customers might be interested in Android efficiency.

 

[Zor Prime]

TI Omap is not available to the public in a power efficient envelope, you need to purchase huge volumes to access particular configurations, so for now I would leave it as it is.

My Pandora has very good battery life, over 10 hours depending what you're doing. Granted, the battery is darn near the width of the unit (pretty flat, though). It OC's reasonably well, too.

 

[MrTeal]

I realize you want to use systems you have available, but once again I have to object to this test. Testing something like the Origen board vs a Brazos system is not testing the efficiency of X86 vs ARM. There are simply far too many other factors involved to be able to look at the numbers and say "There's the x86 decode penalty".

If you want to test the efficiency of the ISA, the closest you could get would be to take an ARM Cortex A8 or A9 CPU (without GPU) as well as an Atom system, build them onto boards where the other components are the same, and then repeat the test. I'd still say that's not a great test for comparing performance/watt between ISAs so much as it is for comparing perf/watt between processors, but it's surely better than comparing a low powered dev board vs a full feature mITX computer.

 

[Smoblikat]

May I ask why a 110F room would be the best? THat seems really hot.

 

[cbn]

Lately there has been a discussion about if x86 can or cannot be more power efficient than ARM.

My null hypothesis is:

ARM is more efficient of x86 in the 1-10W power envelope, due to the lack of a power expensive decode step and missing legacy burden of an ISA not any more adapat to the modern world. This holds especially for the integer operations.

I'm glad you started this thread and are doing this experiment.

I think the situation between ARM and x86 gets even more complicated than "extra decode" when we consider that ARM CPUs are capable of "big.LITTLE" configuration and so far Intel has not presented us with their equivalent of that technology.

 

[Fox5]

I have an AMD Z-01 based system that I'll gladly run benchmarks on, but I don't have UPS or a climate controlled room.

BTW, a Tegra 3 based system will probably beat your chosen ARM chips in performance/watt if your benchmarks scale well over 4 cores.

 

[Idontcare]

I realize you want to use systems you have available, but once again I have to object to this test. Testing something like the Origen board vs a Brazos system is not testing the efficiency of X86 vs ARM. There are simply far too many other factors involved to be able to look at the numbers and say "There's the x86 decode penalty".

If you want to test the efficiency of the ISA, the closest you could get would be to take an ARM Cortex A8 or A9 CPU (without GPU) as well as an Atom system, build them onto boards where the other components are the same, and then repeat the test. I'd still say that's not a great test for comparing performance/watt between ISAs so much as it is for comparing perf/watt between processors, but it's surely better than comparing a low powered dev board vs a full feature mITX computer.

The bolded part, that is the crux of the issue here.

I've no doubt you will be able to benchmark systems, and no doubt that there will be significant differences in the results, but I will have every doubt in any conclusion attempted with such data as it will be intrinsically contrived and not sufficiently controlled/isolated to robustly identify why the differences exist.

This is one of those areas which the markets and economic theory will determine the winner, not the scientists and not in a laboratory.

Think DEC and the Alpha 21264. Superior in so many ways except the one that mattered most (profits).

ARM vs x86 is going to be determined the same way, not because one has an unavoidable decoder overhead penalty or because there is an extra layer of licensing fees involved in the BOM.

 

[ThatsABigOne]

Despite inherent flaws of this test, I am interested.

 

[ncalipari]

How is that a null hypothesis? Wouldn't it be thinking that there are no significant differences in power efficiency between the ARM and x86 architectures?

You can pick whatever null hypothesis you want. The way I stated the problem it's easier to compare the two hypothesis, as it is a one sided comparison, and it is equivalent to state that x86 is more efficient.

Just before this, you wrote that the purpose was to test the efficiency of a full system. How is Android not part of the full system?

Android is greatly optimized for ARM, at the cost of x86 efficiency. It would skew the test at the vantage of ARM. I don't want anyone to say that I cooked the books.

Regarding Android, it was highly optimized for ARM and Intel either was or was not partially or fully successful in performing its own optimizations. From a performance/efficiency perspective, consumers care the results, not history. From a skill set perspective, this is not a level playing field. Chat it up however you want, but Android customers might be interested in Android efficiency.

It's just a question of lawsuits: Google can afford an Oracle lawsuit, Intel instead doesn't want to.

In my opinion they should erase the patent system, but in the short term you either ditch Java or Android.

My Pandora has very good battery life, over 10 hours depending what you're doing. Granted, the battery is darn near the width of the unit (pretty flat, though). It OC's reasonably well, too.

But you can't buy that hardware in low volumes.

May I ask why a 110F room would be the best? THat seems really hot.

Much less hot than your pocket in a warm country. You are interested in worst case scenarios, unless you are part of the marketing team. Heat takes a huge part in efficiency, and that is where ARM really shines.

 

[ncalipari]

So if you guys want this comparison to take off, we need:

1) A java guy to write a benchmark suite and document it.

2) Someone willing to write a script that send the data to our backend, along with a web interface to collect infos

3) People with above said hardware

4) A community-driven process to peer review the methodology

If you know anyone that could help, point them here.

 

[Phynaz]

Heat takes a huge part in efficiency, and that is where ARM really shines.

Here you go posting opinion as fact again.

 

[ncalipari]

Here you go posting opinion as fact again.

Yes, Ohm's opinion.

 

[Zor Prime]

It's not like (99.99%) of anyone here is going to be able to do anything with a SOC design unless it's already integrated onto a board.

TI OMAP systems are easily accessible by the public, there's Beagleboard for one.

 

[jhu]

It's just a question of lawsuits: Google can afford an Oracle lawsuit, Intel instead doesn't want to.

In my opinion they should erase the patent system, but in the short term you either ditch Java or Android.

Is that why Google and Intel are working together to improve Android on x86?

 

[Idontcare]

Yes, Ohm's opinion.

:hmm:

Ohm's law is temperature dependent, not "heat" dependent.

Temperature is heat dependent by way of heat capacity and total mass of the system, but that just means there is an entire range of pre-multipliers involved...including those for which even a more efficient system ends up operating at a higher temperature owing to its lower heat capacity and/or system mass.

 

[ncalipari]

:hmm:

Ohm's law is temperature dependent, not "heat" dependent.

Temperature is heat dependent by way of heat capacity and total mass of the system, but that just means there is an entire range of pre-multipliers involved...including those for which even a more efficient system ends up operating at a higher temperature owing to its lower heat capacity and/or system mass.

http://www.bsideblog.com/images/2008/02/point-at-moon.jpg

 

[ncalipari]

It's not like (99.99%) of anyone here is going to be able to do anything with a SOC design unless it's already integrated onto a board.

TI OMAP systems are easily accessible by the public, there's Beagleboard for one.

The systems I quoted are all boards

Is that why Google and Intel are working together to improve Android on x86?

I believe only in what I can see.

 

[TurtleCrusher]

The testing methodology in this is absolutely awful. You do realize that it wouldn't be anywhere near an absolute or fair comparison to pit x86 against ARM CPUs using java. ARM CPUs have built in Java accelerators (co-processor really). The only test you could possibly make a conclusion with is x86 vs java decoder power efficiency.

This is ridiculous to spend my time replying to such an absurd idea.

 

[Phynaz]

Yes, Ohm's opinion.

Well, ignoring the fact that Ohm's law - which I assume you are referring to - says nothing about "heat", CPU's aren't purely resistive loads, therefore don't fall under Ohm's law.

 

[Blue Shift]

Lately there has been a discussion about if x86 can or cannot be more power efficient than ARM.

How can comparing the power efficiency of current products tell you whether or not it's possible to make an efficient x86 processor?

(Hint: It can't.)