Explain multiple cores to me..........

[Compman55]

I need to better understand multi core cpu's. Does someone have a good link the describes how each core handles data? 2 3.0GHZ cores does not magically equal 6 GHZ does it?

 

[LoneNinja]

Multiple cores can be compared to having multiple processors in your computer. Each core works at it's rated speed and you do not add cores together for total processing speed. Having more cores is like having more workers, more tasks can be done at the same time. Or if the program is multithreaded, both cores can work on the same program and allow it to complete much faster.

 

[Scali]

Think of it as parallel vs serial.

Eg:
We want to transport a load from A to B.
Our truck (core) can only carry 1/10th of the load at a time.
So it will have to drive from A to B 10 times to deliver the full load.

Now if we use multiple trucks, we can carry multiple loads at the same time. A single load won't arrive faster, because the trucks don't go any faster or slower, no matter how many of them there are... But if we use 10 trucks, we only need to drive from A to B once. The trucks don't arrive any faster than the single truck did, but they all arrive at the same time, so we're still done faster.

Clockspeed can be seen as the top speed of the truck.
If you have a truck with 10 times the top speed (okay technically 20 times here, because you'd have to drive back from B to A in this example), you could also do it in the same time as 10 trucks with a lower top speed.

But the problem we have is that we can no longer increase the top speed of CPUs much, while we can put a lot of cores next to eachother on a single CPU, so that's the road we've taken for now.

 

[Compman55]

Ok that makes sense, but lets be realistic, anymore than 8 cores on a single CPU just seems like a waste. At which point are we going to have to increase speed again?

I can;t see the future having 16, 32, 64 cores. I could be wrong.

I am blown away at application loading on a 2.26 core 2 duo vs a single P4 3.0GHz cpu. The slower C2D is like lightning. Would an i7 do basics faster than a C2D? Or is it for intensive things?

 

[Scali]

At which point are we going to have to increase speed again?

Always.
If I can choose between N cores of M GHz or 1 core of (N*M) GHz, I'd pick the single core every day of the week and twice on Sunday.
Multithreaded software is just inherently less efficient than single-threaded software.

I can;t see the future having 16, 32, 64 cores. I could be wrong.

Looks like we're going that way though.

I am blown away at application loading on a 2.26 core 2 duo vs a single P4 3.0GHz cpu. The slower C2D is like lightning. Would an i7 do basics faster than a C2D? Or is it for intensive things?

The C2D is just a lot faster than the P4. Even if you had a single core Core2, it would still make mince meat out of the P4 in most cases. It's just a far more efficient architecture.
As for 'basics'... probably not. In most cases, performance depends a lot more on the amount and speed of your system memory, and the speed of your HDD, than the actual CPU.
I think we've crossed that line a few years ago, so only intensive stuff really matters now. And even then you'd have to look pretty hard.

 

[iNGEN2]

Computer usage today is a lot different than it was 10 years again when most of the foundations of current home computer processor technology was developed. 10 years ago most users operated a single application at a time and frequently waited for it process tasks before closing the application and opening a new one. Today most users operate dozens of applications at the same time. Additionally each individual application performs several, even dozens, of only loosely related tasks simultaneously. Having multiple cores is, with regard to this example, like operating dozens of computers simultaneously.

What I am doing right now is a typical example:

I reading the Anandtech forums in my web browser. But there are five tabs open in my web browser. One in the FS/FT thread, one here in the CPUs & overclocking thread, one open to Google with the results of the last search I did so I can copy and paste sources I wish to cite, one open to the My eBay Watching list, and a last one open to my church's website where I click links every twenty minutes or so to start up the next sermon I have streaming so I can catch up on the what I missed not being able to attend church last Sunday. That stream is in mp3 format so it's being decompressed by MediaPlayer while my virus scanner actively scans the data coming in from each link I click in all five of the tabs I have open as well as any email that arrives in my inbox of the email software I have running in the background.

That's a total of four applications performing 21 processes. Not one of those tasks could be described as "intensive". From my perspective I'm only doing two things; listening to a sermon while browsing a couple web pages. In this type of usage scenario the user, me, would be much better suited by a dozen 1GHz cores, than by a single 4GHz core. Why? Each individual task can be completed by a 1GHz core at a speed indistinguishable to the user from a 4GHz core.

To get an idea of how many cores would benefit you, open Task Manager, click the [Processes] tab. The number of processes counted in the bottom left approximates the number of cores you would benefit by having.

For the flamers out there, I understand my last statement is not technically accurate. However for the purposes of this example it's true to form and facilitates Compman55's grasp of the material.

 

[jaqie]

Think of each core like an employee at a factory. With one, the one person has to do *EVERYTHING* and can be distracted by important tasks which can reduce productivity even when only one iron is in the fire (when only one task is at hand). With two, one can act like a janitor and gopher as the other concentrates on an important task, but efficiency of both of them go down when you have two important tasks to do. Things start getting really productive when you have four employees.

Now, continuing this analogy, some employees are more efficient workers than others, and some also work faster - do not confuse them, this is two different things. MHz (GHz) is how fast they work... There is no totally quantitative means to show a number of how efficiently they work, but there is a way to "Feel" it out, and also get a general idea with CPU (worker) benchmarks.

Now things get a bit muddy... but I think I have a good analogy for this too. Think of hyperthreading as one employee that brought their very well behaved and helpful 12 year old child with them to work... at times it is a disadvantage compared to two employees, but at times it is an advantage compared to one. I actually call this core-and-a-half.

 

[ModestGamer]

Think of it as parallel vs serial.

Eg:
We want to transport a load from A to B.
Our truck (core) can only carry 1/10th of the load at a time.
So it will have to drive from A to B 10 times to deliver the full load.

Now if we use multiple trucks, we can carry multiple loads at the same time. A single load won't arrive faster, because the trucks don't go any faster or slower, no matter how many of them there are... But if we use 10 trucks, we only need to drive from A to B once. The trucks don't arrive any faster than the single truck did, but they all arrive at the same time, so we're still done faster.

Clockspeed can be seen as the top speed of the truck.
If you have a truck with 10 times the top speed (okay technically 20 times here, because you'd have to drive back from B to A in this example), you could also do it in the same time as 10 trucks with a lower top speed.

But the problem we have is that we can no longer increase the top speed of CPUs much, while we can put a lot of cores next to eachother on a single CPU, so that's the road we've taken for now.

ehh they got room. they don't have a cheap cooling solution on the 40+nm chips though.

When the 32nm and the 28nm stuff gets going. You'll see clock speeds shoot back up I expectto see 5ghz 12-20 core cpu's in side of 6 years or less.

Bulldozer from AMD looks promising if you understand the execution advantages it offers. I think Intel is headed down a dead end road though.

 

[Scali]

I wasn't talking about clockspeed so much as I was talking about making the cores faster.
Clockspeed seems to be the only thing that still gradually improves with each process shrink.
But the core logic hasn't really improved much since Core2.

 

[videogames101]

I'm honestly not sure if modestgamer is trolling or not, but anyways...

As software devs make there programs spawn more threads that can be spread across multiple cores you can achieve greater single application performance through the use of multiple cores. Everyday programs don't often use multi-threading because they don't need to, but pretty much all software development nowadays at least considers using multiple threads to take advantage of multi-core processors.

Also, clockspeed does not always improve. Look at the jump from P4 to C2D. It decreased, each cycle just does more work. I'm almost sure i7/i5 does more work per cycle than Core 2.

 

[Scali]

To get an idea of how many cores would benefit you, open Task Manager, click the [Processes] tab. The number of processes counted in the bottom left approximates the number of cores you would benefit by having.

For the flamers out there, I understand my last statement is not technically accurate. However for the purposes of this example it's true to form and facilitates Compman55's grasp of the material.

I don't want to flame you, but this is a pet peeve of mine...
So many people think that the number of processes/threads on a system are somehow related to the number of cores you'd need/want.
Thing is, most threads in a system aren't actually running. Threads are a convenient way for an OS to handle event-based processing.
For example, an application such as Notepad is mostly idle. It only needs to respond to keyboard and mouse input. Windows provides a mechanism where your thread will sleep until a keyboard or mouse event occurs.
The process will then wake up, process the input event, and go back to sleep again. This processing is very simple, and barely takes any CPU time. So you can run thousands of Notepads on a single core (you can never have them wake up at the same time anyway, only one program can receive input at a time, by design... Besides, you can't type that fast).

Considering that most threads in a modern system are this type of thread (goes for browsers aswell... heavy Flash or JavaScript or such may tax the CPU, but most pages just display some text and images, and mainly just wait for user input), they say practically nothing about the number of cores or the CPU power required.
It's the 'worker' threads that matter, but you can't tell which is which just from a task manager overview.

And even that is not that important anymore. For example... in the early days of mp3, Winamp would take up to about 40-50% CPU on my Pentium. Didn't really impair my browsing or editing capabilities, but I did notice with things like compile time. It would have been nice to have a second core to handle the mp3 decoding.
But then I got a Pentium II, which was so much faster at mp3 decoding (thanks to MMX), that the CPU load dropped to about 4-5%.
So now playing an mp3 in the background was negligible, and I no longer needed another core for it.

I think we're at a point where dualcores are just fine for everyday use (Office work, browsing the web, email, playing mp3, video etc), and the system remains perfectly responsive... no noticeable difference with systems with more cores, because two cores are plenty fast at handling all those lightweight threads that are mostly sleeping anyway.
The exception is applications that do heavy processing, and those will generally also be the ones that have been optimzed to use as many worker threads as you have cores, to get the most out of your CPU.

Conversely, you can't really tell from the core loads in task manager whether an application is efficient or not. The most inefficient applications use the most CPU

 

[Martimus]

Now things get a bit muddy... but I think I have a good analogy for this too. Think of hyperthreading as one employee that brought their very well behaved and helpful 12 year old child with them to work... at times it is a disadvantage compared to two employees, but at times it is an advantage compared to one. I actually call this core-and-a-half.

The way I see it is that Hyperthreading is like someone who starts doing a second task when he comes up with a delay in the first task, instead of just waiting out the delay.

 

[Scali]

The way I see it is that Hyperthreading is like someone who starts doing a second task when he comes up with a delay in the first task, instead of just waiting out the delay.

Almost, but not quite.
Since most modern x86 CPUs are already out-of-order, they already try to do a 'second task' when they run into a delay.
Such CPUs have a buffer of pre-decoded instructions to pick from. The CPU knows what each instruction is waiting on (register/memory input, waiting for a suitable execution unit to become available, etc), and picks the first instruction that is marked as ready-for-execution.

The problem here is that CPUs have become so deeply buffered and pipelined that they run out of instructions.
By adding extra instructions from a second thread to the buffer, you at least get two streams of instructions that are independent of eachother. You know that an instruction from thread 0 will not need to wait on results from instructions from thread 1 by definition.

According to Intel, there is no prioritizing going on whatsoever. This means that instructions from thread 0 aren't seen as 'more important' than thread 1. So you never know which thread the next instruction is coming from. The CPU just picks the next instruction in line.

Somehow this is unintuitive to most people. They think the second thread is just 'tacked on' as a second-class citizen, but it's not. It's not 'one physical core' and 'one virtual core'. No, they're both virtualized, and they are both equally important.

 

[jaqie]

Martimus - that's actually not a good analogy because Hyperthreading is a bit smarter than that. The reasons are too technical for me to fully explain at 6 some odd am in the morning just after waking up, but it can do more than that.

~edit~
Thanks, Scali, good job.

 

[mv2devnull]

The way I see it is that Hyperthreading is like someone who starts doing a second task when he comes up with a delay in the first task, instead of just waiting out the delay.

And my vision is slightly different.

However, granularity is an important term.

Do you remember turn-based games that were played by email? You did your turn, and mailed the save to opponent. You did effectively have two cores running the program, one in your computer and another in the opponents PC. But that was sequential processing, and the communication was the slow step.

Step forward to LAN-game. Both cores render bit different views from same data, which they both do update. Communication is still an important lag-inducer.

I've had two cores in a computer for almost 15 years. But at first they were just two separate CPU's. Just like those, the current multi-core implementations do have some dedicated units of their own and some resources shared with the other cores.

Hyperthreading, ... I presume that is has even less "independent" bits and more shared. Therefore, only threads of fine granularity in special applications manage to maximally utilize the additional computation hardware present in the HT core.

Just yesterday I was browsing some server motherboards. Those with four sockets. Each socket could take AMD 12-core CPU. That is 48 cores in single computer. Well, servers do have server uses and prices, and you would need thousands of cores to get into the Top-500 list of supercomputers. There are applications that need all that power. Something that we don't try at home.

 

[Scali]

Hyperthreading, ... I presume that is has even less "independent" bits and more shared. Therefore, only threads of fine granularity in special applications manage to maximally utilize the additional computation hardware present in the HT core.

There *IS* no additional computation hardware present in a 'HT core'.
In fact, a 'HT core' doesn't exist.
It's one physical core, with two decoding frontends... Like a two-headed monster. It has two heads, so it can eat two things at a time, but once it's swallowed it, it all goes through the same body.

That's the thing, they doubled up just enough hardware to process a second thread context, but no actual computational hardware is added.

 

[jaqie]

something that some people don't try at home. 2cpu.com is a place ive been a member of for going on 8 years now, and there are tons of folks there that both need and merely want true multi processor systems. "SMP nuts" like myself. Most of the olden days systems shared a pool of ram, some of the current multi socket boards do not do, including the magny-cours you speak of (between sockets, that is)

 

[mv2devnull]

There *IS* no additional computation hardware present in a 'HT core'.
...
It's one physical core, with two decoding frontends
...
That's the thing, they doubled up just enough hardware to process a second thread context, but no actual computational hardware is added.

Thanks for clarification. By 'HT core' I did mean a CPU core that has the 'HT feature', but should have been more clear.

And ... I knew there are some doubled bits in the physical core, but I never care to remember what bits.

 

[ModestGamer]

no trolling. I make a solid point. As the process's shrink expect to see clocky cycles go up not down. I would not be surprised if we see 12-20 core 28nm chips running at 4-5 ghz inside the next 5 years. Everything is in place to do so. Granted those are likely to be high end workstation CPU's but I would wager a bet we will see 8-10 cores running at clocks of 4-5 ghz easily.

The bulldozer architecture puts forth some interesting ideas on combining the Instruction execution of the CPU with the number crunching of the GPU all in the same processing path.

Should be interesting.

Also comming from AMD is the fusion stuff for next year. 4-6 cores of cpu nd 2-4 core of GPU it seems AMD is far better then this at Intel.

these processors if they work as claimed will have some very huge improvements in PC performance even if the clocks speeds remain the same.


On the software front. the OS should be handling thread deligation. Thats just more bad code from our friends at microsoft. bulldozer from AMD ' at least from early info" looks to remedy this by having the chip handle the thread delegation to some extent.

I'm honestly not sure if modestgamer is trolling or not, but anyways...

As software devs make there programs spawn more threads that can be spread across multiple cores you can achieve greater single application performance through the use of multiple cores. Everyday programs don't often use multi-threading because they don't need to, but pretty much all software development nowadays at least considers using multiple threads to take advantage of multi-core processors.

Also, clockspeed does not always improve. Look at the jump from P4 to C2D. It decreased, each cycle just does more work. I'm almost sure i7/i5 does more work per cycle than Core 2.

 

[Sp12]

You will not see 5Ghz chips anytime soon because of the exponential nature of heat output with increased voltage.

Even with shrinking transistors requiring less and less voltage, it simply is too much heat. Think of it this hypthetical situation (taken from a 2004 Intel IDF presentation, numbers are changed cause I can't remember):

You have a CPU that runs at 2 Ghz and uses 1.0 volts. If you increase the voltage 20%, you can get it to run at 2.6 Ghz (with perfect stability). If you decrease the voltage 40%, you can get it to run at 1.4 Ghz.

Now, if you take two of the downclocked cores, you use 1.2 volts and have a total of 2.8 Ghz of processing power. The single core is only capable of 2.6 on the same voltage. You use less energy and create less heat for the same 'total' performance.

Now, once we get graphene transistors I expect 10+ Ghz on almost no voltage.

 

[JFAMD]

Multithreaded software is just inherently less efficient than single-threaded software.

Wrong.

If your software needs to do the following:

Multiply A*B for result 1
Multiply C*D for result 2
Multiply E*F for result 3
Multiply G*H for result 4
Add results 1-4 for final answer

Would you rather have a quad core and get your answer in 2 cycles or have a single core and get you answer in 5 cycles?

No software is inherently less efficient.
Some workloads may be less parallel.

If you had to do:

Multiply A*B for result 1
Multiply result 1 * 4 for result 2
Multiply result 2 * 7 for result 3
Multiply result 3 * 9 for result 4
Add 16 to result 4 for final answer

Then you'd want the higher clock speed single core.

It all comes down to how much of the application can be done in parallel versus in serial.

 

[deimos3428]

So many people think that the number of processes/threads on a system are somehow related to the number of cores you'd need/want.
Thing is, most threads in a system aren't actually running. Threads are a convenient way for an OS to handle event-based processing.

Well they are related, but who really wants to break out the Erlang tables? Granted it's a mistake to suggest that you need one core per process but the underlying premise is sound.

When we have so many cores that some are never accessed, we have too many. We're nowhere near that point on a modern [Windows] desktop; it's still common to disable background processes or only run a single intensive multithreaded application due to performance. That's going to change in the not-so-distant future: "64 cores should be enough for anybody."

 

[ModestGamer]

they are already made them at 40nm for specific industrial applications. Most of the computing world does not revolve around gamers and PC owners, contrary to there opinions.

You will not see 5Ghz chips anytime soon because of the exponential nature of heat output with increased voltage.

Even with shrinking transistors requiring less and less voltage, it simply is too much heat. Think of it this hypthetical situation (taken from a 2004 Intel IDF presentation, numbers are changed cause I can't remember):

You have a CPU that runs at 2 Ghz and uses 1.0 volts. If you increase the voltage 20%, you can get it to run at 2.6 Ghz (with perfect stability). If you decrease the voltage 40%, you can get it to run at 1.4 Ghz.

Now, if you take two of the downclocked cores, you use 1.2 volts and have a total of 2.8 Ghz of processing power. The single core is only capable of 2.6 on the same voltage. You use less energy and create less heat for the same 'total' performance.

Now, once we get graphene transistors I expect 10+ Ghz on almost no voltage.

 

[Scali]

Wrong.

No, right actually. You just didn't understand that 'more efficient' does not necessarily equal 'faster'.
The 'more efficient' means that it's easier to extract performance from a single thread running on a 2 GHz core than from two threads running on a 1 GHz core each.

If you want the theoretical background to it all, I'll just refer you to Amdahl's law.

 

[Scali]

it's still common to disable background processes or only run a single intensive multithreaded application due to performance.

Uhhh, the point of multithreaded processing is to take advantage of ALL cores.
It's pretty much the same as a singlethreaded application claiming all CPU time on a single-core CPU.
Your system will always become unresponsive because the application is trying to use all your CPU power to get the job done as quickly as possible...
Unless the application is somehow limited to using a certain number of cores, the amount of cores your CPU has, make no difference.