Nothingness said:
It looks like most tests are compiled with -O3, but are lacking -march=native which would enable the use of AVX(2) where applicable. That's unfortunate, but alas a reflection of what happens in real life: given the stupid strategy of Intel to segment parts of their ISA, you can't use such flags by default.
That depends what you want to benchmark. If you want to benchmark CPUs, it is not ok. If you want to benchmark ecosystems, it is ok. But if you do latter you would not only encounter applications without AVX support but also without any ARM support at all. X86 has broad support in server, desktop and mobile (via Android) applications. ARM only has broad support in mobile applications, but nearly no support in desktop and server applications. So if you really want to benchmark the ecosystems, the ARM ecosystem would loose by a huge amount.
The segmenation of the ISA is not a problem at all. Just add CPU-dispatching. Libraries that support AVX usually always contain a path for CPUs that only support SSE and sometimes even for the i686 or i386 subset.
There are several reasons for this in a benchmark.
For one given the limitation of C, where you cannot explicitly express data parallelism, which you need for SIMD, you would rely on auto vectorizations of the compiler.
And second, when looking at general purpose CPU performance, a worse CPU architecture would get away with just providing wide SIMD data-pathes.
In summary i think it is perfectly fine, that a benchmark restricts itself to FPU neglecting any SIMD extensions.
C and C++ have better support for SIMD programming than most other languages. Every major C/C++ compiler like gcc, clang, icpc, msvc does support inline-assembly for x86 and SSE/AVX-intrinsics.
Again it is easy to provide a path for AVX capable CPUs and a path for SSE-only capable CPUs.
To prevent that worse CPU architectures would get away with just providing wide SIMD-data-paths it is necessary to make benchmarks more complex and more realistic. Look at the raytracing part of geekbench. It is a raytracer, any cs student could code. It is a primitive piece of code.
A real professional used raytracer like for instance vray is hundred times more complex. It doesn't run the same code again and again. There is much more diversity in code. Of course wider SIMD-units would also be benefical in vray, because a real raytracer always use SIMD everywhere it is possible. But it also heavenly tests things like branch-prediction, cache-prefetching, etc.
Of course such complex software is not available on iOS. Actually there are almost no real-world applications user run on iOS or Android that really stresses a high-end mobile SoC.
That makes it really difficult to make real-world benchmarks on mobile devices.
I don't think it is a problem if a benchmark is not vectorized, if it is application logic that the average programer would write. But the algorithms geekbench uses don't belong into this category. The algorithms geekbench uses are normally heavenly vectorized and optimized.
Seriously, the argument is about A9X beating some laptop chips according to a single benchmark.
I can understand why many don't believe that, especially when those chips are on similar nodes and one supposedly uses a fraction of the power? Yeah, sure...
It is about not believing that it does. It is about using geekbench as only reference and than thinking the "desktop class performance" claim is true. If I look at wide setting of benchmarks between a8x and core m, even an 80% faster a9x would not match the broadwell core m, let alone the skylake core m.
Thus it is not "desktop class performance", not even "notebook class performance" and even fails to hit the "premium tablet class performance".
It is a good high-end tablet SoC that is sufficient for almost every consumer task normally performed on a tablet, but unsufficient for usage in a productive environment.
Principled Technologies, the makers of WebXPRT, are sponsored by Intel.
http://www.principledtechnologies.com/benchmarkxprt/faq
http://www.vrworld.com/2014/11/03/shades-sysmark-2001-intel-may-webxprt-problem/
Do you really want to defend WebXPRT and keep on spitting at Geekbench?
Sure, unless you have a
real technical argument that specifies how WebXPRT discriminates ARM in favour of x86.
But since it is a HTML 5 & Javascript benchmark it executes the same code for x86 and ARM. And in comparison to Geekbench it actually benchmarks things user typically perform on a mobile device.
So you can't really accuse WebXPRT for using unfair/unrealistic use cases.
But if you find something concrete, I'm open for it.