Question Zen 5 Turin builders thread.

[Win2012R2]

At least you won't need active cooling for them ...

"the DDR5-6000 DIMMs were running rather toasty -- similar to what I've seen with other DDR5-6000/DDR5-6400 AMD and Intel servers too. With this 4U server build I ended up installing two Corsair Vengeance Airflow Memory Cooling Fans (CMYAF) and they made a dramatic difference in the DDR5-6000 RDIMM temperatures. There was a ~30 degree difference and the memory modules not coming anywhere close to throttling." Source: https://www.phoronix.com/review/8-12-channel-epyc-9005

I think I've seen some info (from Dell?) that only select motherboards will support 6000, it does not look like an easy done deal at all. AMD really needs to support MRDIMMs in EPYC 6.

It's really sucky to have a "compatible" socket but MBs require rev 2 to support chip, kind of defeats the purpose.

 

[igor_kavinski]

Its 1.9 ghz, a real dud. I will post something more than this later.

Is your other Turin on its way back or did you manage to test it? Same low speed?

 

[StefanR5R]

Also you must populate a 12 memory slots with 4800 ram.

Maybe a limitation of ES chips? Because Turin is supposed to support 6000 MT/s. It's basically a requirement I think, just like we saw with 9800X3D performing a lot better than 9700X. Zen 5 is very bandwidth starved and really needs it.

ES compatibility quirks aside, how much this is needed depends on the workload of course, and on the core count.

But no motherboard support it yet. I have had all 3, and the specs say 4800 only. One of them has 5600, but not 6000.

Supermicro's specs of H13SSL-N/-NT say that 6000 is supported with board rev 2.x.

Supermicro also announced the H14 series of boards for Turin and DDR5-6000, but so far they only have 4 boards with proprietary form factors and 1 in OCP form factor, that is, none with ATX power connectors and E-ATX case compatibility.

ASUS announced DDR5-6000 capable servers, but I haven't found -6000 capable mainboards on their website.

Gigabyte MZ33-AR1 Rev. 3.x, MZ33-CP1 Rev. 3.x, and MZ73-LM1 Rev. 3.x received 6000 MT/s support, but they are either not shipping yet or backlogged.

ASRock Rack announced Turin/DDR5-6000 capable board but alas only dual-socket boards so far, and they are in proprietary form factors, with the exception of two SSI-EEB boards, both with 8+8 RAM channels only.

At least you won't need active cooling for them ...

"the DDR5-6000 DIMMs were running rather toasty -- similar to what I've seen with other DDR5-6000/DDR5-6400 AMD and Intel servers too. With this 4U server build I ended up installing two Corsair Vengeance Airflow Memory Cooling Fans (CMYAF) and they made a dramatic difference in the DDR5-6000 RDIMM temperatures. There was a ~30 degree difference and the memory modules not coming anywhere close to throttling." Source: https://www.phoronix.com/review/8-12-channel-epyc-9005

Both the RAM and the IMCs pull a lot of power at these speeds. In workloads which are not memory bandwidth bound, perf/W or even absolute performance may increase if RAM is configured to a lower speed.

I think I've seen some info (from Dell?) that only select motherboards will support 6000, it does not look like an easy done deal at all. AMD really needs to support MRDIMMs in EPYC 6.

It's really sucky to have a "compatible" socket but MBs require rev 2 to support chip, kind of defeats the purpose.

ASRock Rack seems to be the lone exception with Turin upgrade possibility on Genoa/Bergamo boards via beta BIOS, but you won't get >400 W and >4800 MT/s that way.

 

[IEC]

I would never bother with a QS or an ES. Ultimately you don't know what is/isn't working or if vendor lock/fuse is tripped. Unless you value your time at $0 you are also ultimately wasting money. Fool me twice...

 

[Markfw]

I would never bother with a QS or an ES. Ultimately you don't know what is/isn't working or if vendor lock/fuse is tripped. Unless you value your time at $0 you are also ultimately wasting money. Fool me twice...

ES/QS chips are never locked. QS is supposed to be a copy of retails chips. ES chips quite often run slower than retail, but all of the functionality is supposed to be there.

 

[Win2012R2]

ES/QS chips are never locked

Perhaps they (ES) are locked for external parties doing their own validation but kept full speed for internal testing?

 

[Markfw]

Perhaps they (ES) are locked for external parties doing their own validation but kept full speed for internal testing?

when he says locked, server AMD chips, once installed in a motherboard are fused internally to only work in the brand motherboard they are installed. Most notable, if they are put in a Dell, they will never work in anything but a Dell system..

 

[Win2012R2]

Most notable, if they are put in a Dell, they will never work in anything but a Dell system..

Ah, got it, yeah, heard that, very very nasty practice - used for real chips too, nevermind ES

 

[DrMrLordX]

ES/QS chips are never locked.

Maybe that changed this generation? Or in a future gen to prevent eBay sales?

 

[Markfw]

Maybe that changed this generation? Or in a future gen to prevent eBay sales?

well , mine works

 

[Win2012R2]

Looks like Turin should be in stock in the UK in January, looking forward to getting one, 9575F supposedly available in first week of January...

 

[Markfw]

OK, Turin done, as good as it gets. It was only running 2.1 under full load. I got a 360 AIO and temps went from 95c (maxed) to 70c but the best I can get is 2.3 ghz under full avx-512 load.

Another issue. Even when I load the drivers for the network ports (H13sssl-nt) win 10 will not recognise them. I was using a USB to ethernet adapter, but I can't that port cleared and faster speed. So I put in a network card. When I do that in slot 1 (right next to the CPU) it won't boot no matter what I tried. I put it in slot 2, and BAM, works perfectly.

Odd stuff but now its done, and even at 2.3 ghz its faster than 3.5 ghz Genoa for avx-512 stuff !

 

[Win2012R2]

even at 2.3 ghz its faster than 3.5 ghz Genoa for avx-512 stuff !

 

[igor_kavinski]

Odd stuff but now its done, and even at 2.3 ghz its faster than 3.5 ghz Genoa for avx-512 stuff !

Makes sense. It must be acting as 4.6 GHz in AVX-512 throughput especially with 12 channels of much needed bandwidth.

 

[StefanR5R]

The particular AVX-512 workload which Mark is running just now is not sensitive to main memory bandwidth. It's a single-threaded FFT with 1.1...1.5 MB worth of coefficients, with countless passes over the same coefficients allocation. That is, the vector execution units can stretch their legs all the way to their power limit.

(At some other times our Distributed Computing team runs FFTs or z-transforms with much more coefficients, and then it's necessary to put several program threads to each task, and to "lasso" all threads of a program instance to the same L3$ domain, otherwise performance would tank as it would be terribly memory bandwidth limited. However, many other Distributed Computing applications are not AVX* accelerated, or only a little.)

 

[xyzzy16]

Has anyone tried DDR5-6400 memory in a Rev 2.0 H13SSL-N or H13SSL-NT with a Turin proc yet? The only case I've seen so far is that Phoronix review.

I just built my first Turin box (H13SSL-NT, EPYC 9355P) and it's currently borrowing a DDR5-5600 stick from another box.

It looks like Supermicro's only tested 16 and 32 GB sticks of DDR5-6400 so far but I'm looking to buy four 96 GB sticks.

 

[Markfw]

I have a 9555 ES, and 6400 ram, but I have not tried upping the memory speed in bios.

 

[xyzzy16]

I have a 9555 ES, and 6400 ram, but I have not tried upping the memory speed in bios.

On H13SSL-N or H13SSL-NT? If so, the default BIOS behavior is to automatically select the speed, which in our cases would be 6000.

Even if you've got it locked down to a lower speed, I'd be curious to know the details on your working RAM (manufacturer/part #/size/speed its running at) so I know what works.

 

[Markfw]

H13SSL-NT Ram is

CORSAIR WS DDR5 ECC RDIMM 64GB (4x16GB) 6400MHz CL32 Intel XMP 3.0 Computer Memory - Black (CMA64GX5M4B6400C32)​

 

[xyzzy16]

H13SSL-NT Ram is

CORSAIR WS DDR5 ECC RDIMM 64GB (4x16GB) 6400MHz CL32 Intel XMP 3.0 Computer Memory - Black (CMA64GX5M4B6400C32)​

Thanks! And I assume you're currently running it at 4800?

 

[Markfw]

Thanks! And I assume you're currently running it at 4800?

Yes, I would love to try it higher. You said you have been as high at 6000 ?

 

[xyzzy16]

Yes, I would love to try it higher. You said you have been as high at 6000 ?

I only currently have one stick of 5600 in the Turin box and its running fine at 5600. It's successfully completed one full round of PassMark's MemTest86 and running another one overnight. Hoping to buy some 6400 tomorrow.

 

[igor_kavinski]

It's successfully completed one full round of PassMark's MemTest86 and running another one overnight.

Why run it on an ECC stick, assuming that's what you have? They fix the errors themselves.

 

[igor_kavinski]

Yes, I would love to try it higher.

Why haven't you???

DO IT.

NOW

 

[Markfw]

Why haven't you???

DO IT.

NOW

It takes about 15-20 minutes every boot, and around 30 minutes or more per failed boot. I need to wait until I have time. Seems like a waste of time, but I can try for 6000.