How much of the task was optimized for the CPU vs the GPU? F@H and other DC tasks are not very useful in directly comparing CPUs and GPUs. They generally run different tasks - some of them are GPU, others are CPU.
How much of the task was optimized for the CPU vs the GPU? F@H and other DC tasks are not very useful in directly comparing CPUs and GPUs. They generally run different tasks - some of them are GPU, others are CPU."bogus"? Hyperbole much? For DC projects, like F@H, the ratio of PPD between the CPU tasks, on a modern Intel AVX-enabled quad-core, versus a modern NV Pascal-based GPU, are indeed on the order of 100x difference (or more!) in favor of the GPU.
You seem to want to paint a picture of GPUs being only marginally faster than CPUs, but that's not accurate, they really are a magnitude (or two!) speedier. Workload permitting, of course. You might just have a poor workload for them.
That's kind of the whole point. To get those "100x speedups", you must optimize your tasks appropriately. You can't just run x64 CPU code on a GPU, and expect it to scale out 100x.How much of the task was optimized for the CPU vs the GPU? F@H and other DC tasks are not very useful in directly comparing CPUs and GPUs. They generally run different tasks - some of them are GPU, others are CPU.
Dataset size in a typical GPGPU application is usually kept limited to not exceed the VRAM. Register size is not much relevant in determining overall performance.
Well, no that isn't the point.That's kind of the whole point. To get those "100x speedups", you must optimize your tasks appropriately. You can't just run x64 CPU code on a GPU, and expect it to scale out 100x.
X=A*BWhat does this have to do with what I said? I merely corrected your initial comment about GPUs having a few hundred GB/s of bandwidth, which is plainly wrong. Even if you discount the registers and the caches, the bandwidth in modern high end GPUs is close to 1TB/s due to compression (for graphics) and other bandwidth enhancing features. Heck, even CPUs have internal bandwidth exceeding 1TB/s. AVX2 and FMA require it.
Also, one has to wonder if register size isn't relevant to determining overall performance in GPUs as you claim, then why do GPUs come furnished with tens of thousands of the damn things?
They compare as single core of an unspecified 1.86GHz Xeon with a 8800GT which doesn't even have DP, using just -O3!In comparison, we compiled the CPU implementation, tMCimg, using the “-O3” option with the gcc compiler and double-precision computation on a single core of an Intel 64bit Xeon processor. of 1.86GHz.
For anyone interested, this thread is inspired by an Intel study and the hilarious response it elicited from NVIDIA.
Looking at the first claim the NVIDIA uses to argue in favor of GPUS, which can be found here, it is obvious that one should be careful whenever somebody claims the kind of speedups.
They compare as single core of an unspecified 1.86GHz Xeon with a 8800GT which doesn't even have DP, using just -O3!
Well, this discussion is strictly in context of compute performance and the how much the actual speedup vs a CPU is, provided both the CPU and GPU is set up accordingly. In the example I listed, the comparison is between a single CPU core with DP, which basically means half the throughput compared to SP. This discrepancy slashes the advantage from 300x to 150x. This is a clear example where the numbers reported are misleading and outright dishonest.Well, you have to see some things in perspective. A cpu core without its caching system and smart prefetching is also limited to main memory bandwidth just as a gpu core is without its prefetching and cache controller. ~50GB/sec (DDR4@3600)max comparing to ~768GB/sec makes the gpu only around 15 times faster in your view but that is wrong.
Take away those things and compare a single cpu core with a gpu core and suddenly you will notice that a cpu is a jack of all trades with sisd with a little simd instructions added, and a gpu core is optimized for simd and a tiny bit of branchy scalar code. But although that simd reads as very limited, the amount of instructions a single gpu core support is still impressive.
The power of a gpu comes from being massively parallel with more than 1000 of those cores and code written to take advantage of that architecture will surpass a cpu core easily in that range of 100 times faster. Take a look at doom2016 and wolfenstein new colossus, those games have engines that make optimal use of the gpu parallel capabilities and does smart prefetching before computing, hiding the latency to get data from system ram to vram and from vram to local internal cache. No cpu can come close.
As long as the beast is properly fed with proper code, the gpu will always be faster than a cpu.
Strip all cache and prefetching and let branchy scalar code execute on a gpu simd unit, it will be slow as hell and then and only then a cpu will surpass it.
The key is to divide and conquer the code. Optimization is key here.
Even CPU numbers are never even close to their theoretical advertised values, IPC is often touted as 6~8 instructions per clocks, real world average IPC? a pathetic 2 instructions! So I don't get your point exactly! Every hardware has a theoretical limit that it never reaches all the time. Even RAM and Hard Disk and SSDs.This is a clear example where the numbers reported are misleading and outright dishonest.
Exactly.But real world code and uses beg to differ. They all point to a massive GPU advantage that is only increasing moving forward.
I can't help but feel this whole thread is nothing more than an elaborate troll. The argument basically boils down to: "GPUs don't reach full theoretical performance, so that means they aren't much faster than CPUs", completely ignoring the fact that CPUs never do, or anything else for that matter.Even CPU numbers are never even close to their theoretical advertised values, IPC is often touted as 6~8 instructions per clocks, real world average IPC? a pathetic 2 instructions! So I don't get your point exactly! Every hardware has a theoretical limit that it never reaches all the time. Even RAM and Hard Disk and SSDs.
It's hilarious to see him push Intel "studies" after they lost billions on the failure that was Larrabee. They were telling us the thing was going to make GPUs obsolete because it got "moar FLOPS with ray-tracing". Now we have Intel licensing AMD GPU tech. In other-words the complete opposite of their lunatic Larrabee ramblings.
OP, why do you think there's a GPU mining craze if CPUs are just as fast for crypto compute tasks?
Why use silly metrics like IPC when you can just measure time to completion? Because then you'll be caught doing something silly to hide the fact that GPU performance is grossly overestimated in most real-world applications.Even CPU numbers are never even close to their theoretical advertised values, IPC is often touted as 6~8 instructions per clocks, real world average IPC? a pathetic 2 instructions! So I don't get your point exactly! Every hardware has a theoretical limit that it never reaches all the time. Even RAM and Hard Disk and SSDs.
HPC, supercomputer, AI machines and servers all are using GPUs because they ARE several magnitudes faster than CPUs, doesn't matter what Intel's pathetic reasoning is. They are losing the battle that's why they are trying to belittle the difference. But real world code and uses beg to differ. They all point to a massive GPU advantage that is only increasing moving forward.
Ah, so you're ignoring the facts because it came from Intel? In that case are you buying into NVIDIA's weak argument that 'look GPUs are so easy that even MIT undergrads can code for them, but Intel has to hire experts to make their CPUs work'. That NVIDIA response was plain stupid.I can't help but feel this whole thread is nothing more than an elaborate troll. The argument basically boils down to: "GPUs don't reach full theoretical performance, so that means they aren't much faster than CPUs", completely ignoring the fact that CPUs never do, or anything else for that matter.
It's hilarious to see him push Intel "studies" after they lost $billions on the failure that was Larrabee. They were telling us the thing was going to make GPUs obsolete because it got "moar FLOPS with ray-tracing". Now we have Intel licensing AMD GPU tech. In other-words the complete opposite of their lunatic Larrabee ramblings.
Likewise, some of Intel's Coffee Lake press release speedup claims were based on increased core count when in reality almost no software gets perfect scaling and saturates all cores 100%.
OP, why do you think there's a GPU mining craze if CPUs are just as fast for crypto compute tasks?
Time of completion of what? Even going by your FLOPs metrics, CPUs don't even come close to their theoretical limits. Most gains from multi core CPUs are negligible, whether in games, video editing, photo editing, and even many 3D applications. As CPUs fail to scale up spectacularly. While GPUs scale much much better.Why use silly metrics like IPC when you can just measure time to completion?
NVIDIA gave plenty of links for real workloads showing 100x speed ups compared to CPUs. Intel gave none. They ran unknown code on an old GPU and declared it only 14x faster! When it comes to evidence, Intel showed none, while NVIDIA showed it all.In that case are you buying into NVIDIA's weak argument that 'look GPUs are so easy that even MIT undergrads can code for them
No, my question is, provided that you have a workload that can be parallelized on both CPUs and GPUs, and using all available optimization methods on both of them, what is the difference in performance between the two?If you're saying that not all code is embarassingly parallel, and therefore, code ported directly from CPUs to GPUs, will result in underwhelming performance speedups, I agree with you.
But if you take code that IS embarassingly parallel, and optimize that code for GPUs, and let it fly, then completion times alone, suggest that GPUs ARE 100x faster than CPUs, or more.
Edit: My PrimeGrid PPS Sieve WUs, take 17 hours on a CPU core, and like 18 minutes on a GPU (with 2048 cores, an RX 470 4GB).
Ugh...this is in essence what the first NVIDIA-linked "study" does to claim 300X performance.NVIDIA gave plenty of links for real workloads showing 100x speed ups compared to CPUs. Intel gave none. They ran unknown code on an old GPU and declared it only 14x faster! When it comes to evidence, Intel showed none, while NVIDIA showed it all.
That's simply wrong. CPUs can scale just as well, provided of course you know your workload. Time to completion refers to refers to the actual way people measure performance, like in this test.Time of completion of what? Even going by your FLOPs metrics, CPUs don't even come close to their theoretical limits. Most gains from multi core CPUs are negligible, whether in games, video editing, photo editing, and even many 3D applications. As CPUs fail to scale up spectacularly. While GPUs scale much much better.
LOL! The test isn't even optimized well for GPUs to begin with, It is stated right there in the first section. I don't even know why you are so hung up on it.Time to completion refers to refers to the actual way people measure performance, like in this test.
If you don't like the first comparison, switch to the others then. GPUs are still far faster than CPUs. Keep in mind this is 2010 we are talking about. By now a Titan V can be 1000x better than a common CPU.Ugh...this is in essence what the first NVIDIA-linked "study" does to claim 300X performance.
Aha! And here lies the crux of the matter, use a suitable workload and your hardware can push to the limits. Use a non suitable workload and your hardware acts like a turd.That's simply wrong. CPUs can scale just as well, provided of course you know your workload.
Well, this discussion is strictly in context of compute performance and the how much the actual speedup vs a CPU is, provided both the CPU and GPU is set up accordingly. In the example I listed, the comparison is between a single CPU core with DP, which basically means half the throughput compared to SP. This discrepancy slashes the advantage from 300x to 150x. This is a clear example where the numbers reported are misleading and outright dishonest.
If you looked at the Intel study, then they show that the geometric mean of the speedup that a GTX 280 gives over an i7 960 is 2.5x, with the best case scenario being 14.9x. This is a long way off the >100x speedups that NVIDIA especially likes to claim. This is directly tied to the fact that the touted GFLOP/s numbers are never going to be achieved running real applications.
For some other results from a different domain, have a look at this thread.
For proper SIMD gains you need to design the data structures accordingly and think about that stuff from the ground up, not as an afterthought
....
And still, even after all this happened, a better algorithm, or better data structures or less cache misses, still give you more gains than SIMD. SIMD itself does not guarantee that your code is any good. Any non-SIMD code can kick SIMD code’s ass any day of the week if SIMD code is clueless about everything else.
...
EDIT: I forgot something. Contrary to what people also claim, PhysX works just fine on consoles and it is a multi-platforms library. That is, writing SIMD is not as easy as hardcoding a bunch of SSE2 intrinsics in the middle of the code. It has to be properly supported on all platforms, including some that do not like basic things like shuffles, or do not support very useful instructions like movemask. Converting something to SIMD means writing the converted code several times, possibly in different ways, making sure that the SIMD versions are faster than their non-SIMD counterparts on each platform - which is not a given at all. It takes a lot of time and a lot of effort, and gains vary a lot from one platform to the next.
That test clearly states the situations in Ansys Fluent which are favorable for GPUs and which are not. In the situation where GPUs can benefit to a significant degree, it is also shown that throwing in more CPU cores is equally viable. You have zero understanding of what the test showed, which is clear from your arguments.LOL! The test isn't even optimized well for GPUs to begin with, It is stated right there in the first section. I don't even know why you are so hung up on it.
Actual hard numbers please, in a fair comparison? You're acting as if CPUs haven't gone anywhere since 2010.If you don't like the first comparison, switch to the others then. GPUs are still far faster than CPUs. Keep in mind this is 2010 we are talking about. By now a Titan V can be 1000x better than a common CPU.
Again showing your lack of understanding. Take the best example of a highly parallelizable compute task, like matrix multiply, on a modern Intel CPU, say the Xeon Gold 6148 with the latest vector SIMD instructions, use MKL to write optimized code, choose a suitable problem size that utilizes the main memory, and report what GFLOP/s you get.Aha! And here lies the crux of the matter, use a suitable workload and your hardware can push to the limits. Use a non suitable workload and your hardware acts like a turd.
When it comes to desktop scaling, GPUs outpace CPUs by orders of magnitudes, they scale massively better in gaming, Video and Photo editing, 3D Rendering, Video acceleration, Crypto, Folding, AI, and dozens of other scientific applications. While CPUs generally SUCK there. When was the last time you saw a game that successfully utilized even 6 cores?
This is a typical example of how important the implementation of an algorithm is. For to be fast on a gpu it must be highly parallel and i mean make use of hundreds or thousands of cores.
We do not know what the code is ansys runs. It is not wise to just take this example and say gpu are slow.
If the code was thought up from the ground up to run best on a cpu, it will never run good on a gpu because of the reasons i mentioned in my previous post.
See my last response to Muhammed in the previous post. You're talking about generalities while I'm talking about a specific thing. Linear algebra. Both CUDA and Intel MKL can accelerate matrix operations, which is one of the best examples of a highly parallel workload.For example, the cpu has sse2 or avx. It is possible to write code that runs awful even while using sse2 or avx.
Code has to be specially written for it. Optimization.
You cannot just take code and compile it for a different architecture and expect it to be hell of a lot faster just like that.
Sometimes it happen but most often not.
Which actually means the code is not optimized well for GPUs. Being favorable to GPUs in some cases doesn't in anyway mean the code is even 30% suitable for GPUs. It's obviously not even 20% suitable if adding more CPU cores is equal to adding more GPUs.That test clearly states the situations in Ansys Fluent which are favorable for GPUs and which are not. In the situation where GPUs can benefit to a significant degree, it is also shown that throwing in more CPU cores is equally viable.
Take a look at Folding@Home for example, GPUs vastly outpace CPUs there. F@H is one of the best examples of a highly parallelized code optimized for both GPUs and CPUs.Again showing your lack of understanding. Take the best example of a highly parallelizable compute task, like matrix multiply, on a modern Intel CPU, say the Xeon Gold 6148 with the latest vector SIMD instructions, use MKL to write optimized code, choose a suitable problem size that utilizes the main memory, and report what GFLOP/s you get.
Actual hard numbers please, in a fair comparison? You're acting as if CPUs haven't gone anywhere since 2010.
See my last response to Muhammed in the previous post. You're talking about generalities while I'm talking about a specific thing. Linear algebra. Both CUDA and Intel MKL can accelerate matrix operations, which is one of the best examples of a highly parallel workload.
When programming GPUs we know that we typically schedule many 1000s of threads and we also know that we can further organize them in many tiles of threads.
Aside: These concepts also exist in other programming models, so in HLSL they are called “threads” and “thread groups”. In CUDA they are called “CUDA threads” and “thread blocks”. In OpenCL they are called “work items” and “work groups”. But we’ll stick with the C++ AMP terms of “threads” and “tiles (of threads)”.
From a correctness perspective and from a programming model concepts perspective, that is the end of the story.
The hardware scheduling unit
However, from a performance perspective, it is interesting to know that the hardware has an additional bunching of threads which in NVIDIA hardware is called a “warp”, in AMD hardware is called a “wavefront”, and in other hardware that at the time of writing is not available on the market, it will probably be called something else. If I had it my way, they would be called a “team” of threads, but I lost that battle.
A “warp” (or “wavefront”) is the most basic unit of scheduling of the NVIDIA (or AMD) GPU. Other equivalent definitions include: “is the smallest executable unit of code” OR “processes a single instruction over all of the threads in it at the same time” OR “is the minimum size of the data processed in SIMD fashion”.
A “warp” currently consists of 32 threads on NVIDIA hardware. A “wavefront” currently consists of 64 threads in AMD hardware. Each vendor may decide to change that, since this whole concept is literally an implementation detail, and new hardware vendors may decide to come up with other sizes.
Note that on CPU hardware this concept of most basic level of parallelism is often called a “vector width” (for example when using the SSE instructions on Intel and AMD processors). The vector width is characterized by the total number of bits in it, which you can populate, e.g. with a given number of floats, or a given number of doubles. The upper limits of CPU vector widths is currently lower than GPU hardware.
So without going to any undesirable extremes of tying your implementation to a specific card, or specific family of cards or a hardware vendor’s cards, how can you easily use this information?
Avoid having diverged warps/wavefronts
Note: below every occurrence of the term “warp” can be replaced with the term “wavefront” and the meaning of the text will not change. I am just using the shorter of the two terms .
All the threads in a warp execute the same instruction in lock-step, the only difference being the data that they operate on in that instruction. So if your code does anything that causes the threads in a warp to be unable to execute the same instruction, then some threads in the warp will be diverged during the execution of that instruction. So you’d be leaving some compute power on the table.
You're still not paying attention to what I'm saying. Yes the general GPU architecture and programming model means that you have to have a task that is amenable to parallelization under that model, to ensure maximum utilization of resources at all time.I am not writing about generalities.
It is a perfect explanation why code sometimes is not faster and when it is not.
And there is more of course :
On a gpu to get close to that theoretical number, you must write an algorithm that allows all threads in a warp or wavefront to be fully active with as little if then else statements as possible.
When some threads take the if and the other the else statement, during the execution of the instruction some threads from a warp/wavefront can not complete and have to wait.
Then when these thread can execute, the other threads have to wait.
From 32 threads, not all are in use all the time.
This is what we see in games as well, when a game cannot make full use of all threads, be it warp or wavefront the fps will be lower because not all simt (simd) units can be utilized properly.
That is where profiling software comes in.
I am sure Nvidia and AMD provide profiling software that is not just suited for graphics alone.
Read this from a gpu programmer.
https://blogs.msdn.microsoft.com/nativeconcurrency/2012/03/26/warp-or-wavefront-of-gpu-threads/
It is about graphic programming, but the same applies for cuda or opencl.