Hammer effective bandwidth for the memory?

[BumJCRules]

I posted this on THGC, Toms Hardware Guide Community, and I was wondering if anyone here knew the answers to my questions.

Background...

"The Hammer micro-architecture incorporates a dual-channel DDR DRAM controller with a 128-bit interface capable of supporting up to eight DDR DIMMs (four per channel) as seen in the Hammer micro-architectural diagram in Figure 2. The controller will be initially designed to support PC1600, PC2100, and PC2700 DDR memory using unbuffered or registered DIMMs. This translates into available bandwidth to the processor of potentially up to 5.3GB/s with PC2700 memory." - AMD Eighth-Generation Processor Architecture Page 4.

On page 5 it goes on to state...

"The integrated Hammer memory controller has an even more dramatic effect when powering multiprocessing systems. The controller results in an outstanding advance in x86 system architecture scalability by enabling ?glueless? multiprocessing where the available memory bandwidth to the system scales with the number of processors. In Figure 3, an example of a four-processor multiprocessing system is shown. In this configuration, the system is able to support up to 32 DIMMs capable of delivering an extraordinary 21.3GB/s of available memory bandwidth to the system with PC2700 memory."

The reason is that on a 4-way system each processor die can access any DIMM. So 2 channels x 4 sets of channels x 500/3 (~166.667MHz) = 21.333Gb/s.

The only thing I am not sure on is how limiting the Hypertransport for a multiprocessor system will act. I.E say a 4 way system. Hypertransport is a bi-directional, 16-bit, 1600MT/s (Mega-transfers per second), 3.2GB/s data-pathway. See page 7 for a block diagram.

On page 5 of the AMD HyperTransport? Technology I/O Link - A High-Bandwidth I/O Architecture, it shows that the Hypertransport bus is 32-bits wide with a maximum of a 1.6GHz or 1600MHz signal. (1600MHz x 4 Bytes = 6.400GB/s. Now that is one direction. Bi-directional would be 12.8GB/s.

My questions are these...

1. Which is it in the core? 16-bit or 32-bit?

2. Does one use only one direction or both to figure the bandwidth for Hammer using HT?

[rant on]
Also, in the "AMD?s Next Generation Microprocessor Architecture" which was presented by Fred Weber back in October of 2001, one point was "Bandwidth and capacity grows with number of CPUs." This cannot be... But that is another story. [rant off]

I don't know what to exactly believe.

For a single chip setup, the peak bandwidth will be 5.3GB/s. For a multiprocessor system, say a 4-way system, it is peaked out at whatever HyperTransport will allow or what the memory will allow, whichever is smaller or some combination of the two together. That would be... 6.4GB/s (in one direction) x 2 HT data paths = 12.8GB/s for the Hypertransport. The memory would be memory bandwidth of 5.3GB/s per CPU die. (4 CPUs x 5.3GB/s = 21.2GB/s) That is the number that AMD shows but I am not sure it can reach that memory number.

I think it will be something like this.

5.3GB/s for each CPU plus what it can access from the other CPUs' DIMMs. So I think it would be 5.3GB/s (CPU direct)+ 12.8GB/s (Hypertransport) = 18.1GB/s.

Now again this is only my assumption based on what I have read and understand. So if someone out there has a better understanding or has a spin I am not seeing yet, please point it out.

 

[Evadman]

more info on hammer memory archetecture and some diss. here in HT can be found here:


1
2
3

 

[BumJCRules]

Thank you for the links.

However they don't talk about the details.

I am talking to Cheeta05r over at Hard Forum and we are not sure.

And so far no one at Tom's has been able to shed some light on the subject.

 

[Peter]

What operating systems with such a loosely coupled RAM architecture need is CPU affinity of memory allocations. Tis is to make sure that tasks and threads that run on a certain CPU have their code and data reside in memory that is local to this CPU.

regards, Peter

 

[Eskimo]

My questions are these...

1. Which is it in the core? 16-bit or 32-bit?

Cores communicate with eachother through 16bit coherent HT links.

2. Does one use only one direction or both to figure the bandwidth for Hammer using HT?

Both

See here for block diagram

 

[BumJCRules]

"Cores communicate with eachother through 16bit coherent HT links."

However in a dual+ system there are two HT links in each processor core. If you had two cores, there are two on each and thus they can talk to each other using both. If you use more that two processors in the setup they still have two. Chip 1 can talk to chip 8 via 3 jumps on the HT subway and vise versa from any other chip. That is the longest path. Some are 1 HT jump away and some are 2 away.

So like I said, is the programming language written for use of only one HT or both? 16bit or 32bit transmissions?



My question was... "2. Does one use only one direction or both to figure the bandwidth for Hammer using HT?"

You said both...

I just don't see how you could use both directions for bandwidth. A transmission goes from point a to point b not to point a to b to a.

For read an writes to another's CPU's memory I don't think so. For sending and recieving info at the same time pehaps. But for bandwidth calculations you can't use both. If I am incorrect please correct me. But if you do please send me somewhere or quote something to prove it to me. Sorry to be an A Type personality here, but I want to learn more about systems and sub systems.

 

[cheeta05r]

I posted this on the Hard Forums and Bum_JCRules has allready read it, but some other people may want to read the conclusions I came to. The questions that are still unanswered are:
1. Does one use only one direction or both to figure the bandwidth for Hammer using HT? [Would like a concreate yes or no]
2. The memory bandwidth and what the bandwith is across the whole system.

Originally posted by Bum_JCRules

1. Which is it in the core? 16-bit or 32-bit?

I am not sure, but I believe the dual configuaration you looked at with 3.2GB/s HyperTransport is a ClawHammer DP set up. The ClawHammer has less pins so it would make sense that it only has 16-bit HyperTransport. The 4 processor config picture shows 6.4GB/s HyperTransport bandwidth. I believe this is the SledgeHammer, more pins, makes sense that it is 32-bit.
2 Processor Value Server 3.2GB/s
4 Processor Configuration 6.4GB/s

Originally posted by Bum_JCRules
2. Does one use only one direction or both to figure the bandwidth for Hammer using HT?

In the pictures above it says "6.4GB/s coherent HyperTransport." I looked up coherent on dictionary.com and this is the definition:

Physics. Of, relating to, or having waves with similar direction, amplitude, and phase that are capable of exhibiting interference.

The "similar direction" part leads me to believe that the 6.4GB/s is in one direction. Making it 12.8GB/s back and forth. I am not really sure about this that is why question 1 still stands.

 

[zephyrprime]

How does memory access work in a hammer multiprocessor setup? If one processor1 tries to access memory that's physically connected to processor2, does processor1 have to communicate with processor2 and tell processor2 to fetch the memory and then processor2 communicates the data back to processor1?

 

[BumJCRules]

"How does memory access work in a hammer multiprocessor setup? If one processor1 tries to access memory that's physically connected to processor2, does processor1 have to communicate with processor2 and tell processor2 to fetch the memory and then processor2 communicates the data back to processor1?"

It works in two ways.

The first access is direct from processor 1's memory controller to the memory on that processor's memory bus.

The second is where processor 1 look to processor 2's memory controller to find some info. If it is not there it goes to the next until it finds what it is looking for. However, I don;t know what kind of memory monitoring there will be. Over such a large array there will need to be some form of managment or there will be a lot of misreads and wiswrites. Big delay problems could occur. However they probably have solved this dilema. That is just speculation, however the process that it would take in a multi-processor setup would be as I discribed.

 

[cheeta05r]

BumJCRules I think I found some pics that will help. They are on this website, Muropaketti.
http://www.muropaketti.com/artikkelit/sekalaista/amd_hammer/11.jpg.
http://www.muropaketti.com/artikkelit/sekalaista/amd_hammer/10.jpg.

HyperTransport Slide 1
[*]HyperTranport Technology buses have two unidirectional point-to-point links.

>The links can e 2-, 4-, 8-, 16, or 32-bits each direction.
>HyperTransport links have a data rate up to 1600 Megabits/second per pin-pair (800MHz clock).

• E.G.. 4 bits each way give up to 1.6 GB/s total bandwidth

• E.G.. 8 bits each way give up to 3.2 GB/s total bandwidth

• E.G.. 16 bits each way give up to 6.4 GB/s total bandwidth

• E.G.. 32 bits each way give up to 12.8 GB/s total bandwidth

* Packets are multiples of 4-bytes in length.
* Serial link with commands, addresses and data use the same bits.



HyperTransport Slide 2

• One, two, or three links.

• 2, 4, 8, 16, or 32-Bits full duplex

• Up to 6.4 GB/s bandwidth per link

• 19.2 GB/s aggregate Bandwidth

 

[StumbleBum1]

bump