Metric for CPU Performance

[hotdogchef]

Several years ago I remember thinking that the frequency measurement as a measure of a CPU would likely fade away as multi processors became more commonplace and frequency was not necessarily a true measure of a CPUs performance. Today, as I am making a new PC and refreshing on some of the details I am shocked to find out that another seemingly relevant metric, specifically FLOPS, is still not widely advertised as a metric of a CPU. It is still a buried number difficult (impossible?) to find unless you dig. Is there some reason FLOPS or some other metric is not used or it still true that frequency is a good assessment of performance? Does it not all come down to the FLOPS that a CPU can do, regardless of advertised specific speeds for other components of the CPU?

 

[jhu]

Not sure how relavent that would be. It's easy to calculate theoretical flops. For example, a Haswell core would be 4 64-bit flops/cycle or 8 32-bit flops/cycle (eg, 12.4 GFLOPS/24.8 GFLOPS at 3.1 GHz for double and single floats, respectively). And then there's actual sustained flops. This is probably more relevant for scientific computing, but a bit less relevant for gaming. It is also highly code dependant.

Ulimately it does come down to performance. The theoretical peak flops of a pentium 4 @ 3.8 GHz (7.6 GFLOPS double, 15.2 GFLOPS single) is higher than a core2 @ 2.5 GHz (5 GFLOPS double, 10 GFLOPS single). But we all know that the core2 is faster in the real world than the pentium 4 mainly because real world code branches a lot and the pentium 4 has a much higher penalty for incorrectly predicting which branch to take.

 

[Cerb]

http://en.wikipedia.org/wiki/Random-access_memory#Memory_wall

Modern CPUs take a long time to get out to memory, on the order of 60+ns for a random (unpredicted) access, when each clock cycle is <1ns (3GHz is 1/3ns).

While frequency hasn't gone up much, effective performance per clock has, quite a bit. But, all that means is that the complexities that they thought they'd have to worry about to make single-issue or dual-issue 10GHz CPUs, way back when, are having to be dealt with in 4-issue to 8-issue 3GHz CPUs, today.

Anybody can add RAM bandwidth and get FLOPS in silly benchmarks. Not everybody can make use of them in applications that do real work, or even benchmarks that try to simulate real work. The right data needs to be in L1, or at worst, L2, before it is requested, to keep up with the CPU's data crunching abilities, making computation itself quite cheap, in the scheme of things.

As a secondary effect, cache coherency implementations, and on-chip CPU to CPU communications, have a real impact on performance, not merely what's inside of each CPU. For example, AMD's recent CPUs are only just up to the single-threaded performance of their 2009 CPUs, yet leave them behind in modern multithreaded games by quite a margin.

Outside of HPC, where spending a ton of time and money to tailor a workload to a single computer system is not uncommon, FLOPS are more of a diagnostic measurement than a useful performance metric.

 

[Borealis7]

the metric of a CPU is whatever work you do with it. if your work is running a complex simulation, then measure that simulation's time and that would be your metric. if you're playing a game, measure FPS etc'.

 

[BrightCandle]

There is no simple metric for a CPUs performance than we can use as a proxy for performance. The design and implications of that design are so complex as is the real world software that its impossible to sum it up with a single number or small grouping of them. Even todays testing is extremely limited compared to the software available.