Question Genoa builders thread.

[Markfw]

Yes you are correct - it is H13SSL-NT with 10 gbits! Good opportunity to kick Broadcom networking here - Intel far more compatible, its 10 gbits SFP card was detected during install just fine.


It's the last Windows server OS that has NO per core licensing, so when using it with 96 cores it's pretty cost effective - the price of current Windows core packs will be close or higher to cost of MB+CPU+memory. Windows 2012 R2 supports NVMes, can in theory support up to 255 cores too - so savings might be even higher with future processors.

Having said all that the next destination will be Linux - that installed perfectly with default BIOS settings...

---

One important thing to add here - despite lack of chipset drives Windows 2012 R2 seem to be able to lower clocks when idle to reduce power usage, so it does not have to run at max level all the time, in fact this is where one of the glitches seem to be (it is present also on supported EPYC 7002s) - you have to put system into Balanced Mode in order to get clock boosting, if put into Performance then for some reason it will stick to BASE frequency, not a biggie.

So you are using these for what they were meant for server use ! Normally we don't hear about this, as it is left to the data centers. I and the people that I know of, use it for distributed computing. I have 7 9554's, t 9654 and a Turin, all in my house, and all doing DC work on gigabit network.

 

[StefanR5R]

"low activity workload" — whatever this means; this may quite possibly exclude SMT usage for example (not to mention vector arithmetic)
"and other conditions" — low core temperatures for example?

PS, perhaps the "other conditions" also include that fewer than the possible 12 RAM channels are populated. Perhaps just one? Reduces IMC power usage and at the same time forces cores to idly wait for RAM accesses instead of doing real work...

 

[Win2012R2]

Try CDR-NIC 1.62. Version 1.70 appears to lack Windows 2012 support.

Thank you - we've got plenty of Intel 10 Gbits with SFP ports which are more common in our switches, they are very solid cards

I am just keeping servers busy now to test stability, tomorrow will do more of BIOS tweaking, if 240W figure correct then 96 cores might just not be getting enough juice to go higher than 3 ghz, will test it with our own workloads too, will report later...

PS, perhaps the "other conditions" also include that fewer than the possible 12 RAM channels are populated

We've got 6 populated right now, maybe that reduced power in IO die? It should not lowered max core freq then - if anything more power will be available to feed cores rather than IO.

 

[marees]

DeepSeek R1 on AMD EPYC with 768gb ram (no GPU)

DeepSeek R1 (inference) setup without any (costly) GPU

2x AMD EPYC 9004 or 9005 CPU
768gb RAM
(Speed: upto 7 tokens/sec)

https://twitter.com/x/status/1884244369907278106

some quick calculation:

💡6-8 tokens/s
💡400W power unit
💡~$0.176 per kWh

To generate 1m tokens:

~40h * 0.4kW --> 16 kWh
16 kWh * $0.176 --> $2.82

$2.82 in electricity costs to generate 1m tokens. Not bad!

https://twitter.com/x/status/1884320857331490849

 

[StefanR5R]

What now? You'll have either 2x EPYC 9000 or 400 W, but not both.