Question Genoa builders thread.

[Markfw]

OK, so I started a post in the distributed computing forum outlining my 9654 Genoa build for DC projects. Well, I had someone else came in to discuss their Genoa build. So there may only be 2 of us, but this is where the thread should live. So, anyone building a Genoa system, or comments on a Genoa build is welcome here. I will add a few pics below, but not to duplicate what I did in the DC forum. Currently waiting on the retail 9654 due Sunday. I have all the other pieces, except the GOOD heatsink. I have one that will allow me to finish the build, buts only 320 watt rated, and LOUD. Current config:
9654 96 core, 192 thread EPYC CPU
192 gig DDR5 4800 ECC RDIMM memory.
2 TB Corsair MP600 PRO XT NMVE
TiTan-V video card.
RM 1000 power supply.

Once running, I will try and get some kind of windows install for benchmarking, but Linux will be the main OS for running.

 

[Markfw]

Damn ! You won't believe it, but there are now 3 of us that will be posting about our Genoa builds !

 

[jonperron09]

Here is my current config

Case : Rosewill RSV-L4500U (with 8 fans)
MB : Supermicro H13SSL-NT
CPU : EPYC 9374F
Heatsink : 400w rated model from Alibaba
Memory : 12x Micron DDR5-4800 64GB ECC RDIMM
SSD : 2x WD Black SN850X 4TB
PSU : FSP Twins Pro 900W

I should start the build in like 3-4 weeks when I get everything.

 

[Hitman928]

You all are crazy and I love it!

 

[igor_kavinski]

Big boys with the big guns

Wish I could join the club

 

[A///]

Very clean build, Mark. I'm curious what the other 2 do with these because you've already told me. I should do a costs analysis or hire my accountant to do it the next time I see to upgrade builds after this June one. May be cheaper to spend the 12K in 3-5 years than smaller mainstream builds.

 

[Thibsie]

Wish I could get a simple competent rack server for my (public) school as too show the kids what it is what one can do with it.

I love teaching things without the hardware 'cos we can't afford it. Huh...

Nice to see a Genoa builders thread

 

[vityan666]

Oh well, I'll post my planned config as well.

Case: Corsair 1000D SuperTower and 8 fans
MB: Supermicro H13SSL-NT
CPU: Epyc 9274F 24 core
Cooler: The larger heatsink with 2x120mm fans, manufactured by chineese CoolServer company.
Memory: 6x or 8x 32GB Micron DDR5 RDIMM 2R modules.
PSU: Current Corsair AX1000 or I'll buy Seasonic Prime TX|PX-1600|1300.
VGA: Gigabyte Windforce Radeon 7900XTX OC.
Raid Controller: Areca 1883ix-12 RAID controller with 8GB DDR3 ECC DIMM buffer + BBU + Cooler card with 3 fans for regular tower, otherwise expander chip and CPU would consistently overheat.
Temps are very good with this fans card(Dropped by 20-30 degrees celcius).
UPS: APC SMT2200I Line Interactive 2.2KVA

Storage:
1) 4x DC HC530 14TB Enterprise HDD SATA-6,512e in RAID-6 on Areca
2) 1x Seagate Nytro 3532 2.5" 3.2TB SAS 12Gb/s 3D eTLC Enterprise SSD, 3DWPD as JBOD on Areca
* Used for main system and Hyper-V storage volume as reliable and relatively fast storage device.
3) 1xSamsung 860 Evo 2TB SATA SSD to store games requiring faster disk access.
* Don't laugh ppl, I still store most of my games on HDD's RAID and it's still fine in most cases(RAID card's read ahead and large buffer compensates their speed very well)
4) 1xSeagate Exos X18 18TB Enterprise HDD SATA-6,512e to allow fully backup RAID-6 data in case I'll want to recreate RAID volumes with different sizing, which I'm thinking of for about a year now.
5) 1xMicron 9400 Pro 7.68TB Enterprise SSD U.3 1DWPD - Will replace the disk (2) as main storage(+ extra Games volume )

Extras:
1) Intel X710-2T 10G Ethernet Controller
* I'm not sure I'll move it to the new config, or reconfigure to use integrated Broadcom chips.
2) USB-A PCI-E card with 5 USB 3.2 Gen 2 and USB hub from Ugreen
3) Creative Sound Blaster X4 Hi-Res External USB sound card.
4) VGA-Out to DP-In Adapter with USB power to pair BMC with modern monitor, at lower res.
5) 1xPCI-E 4.0 Riser cable to replace current PCI-E 3.0 one, if it will be needed.
6) 1xPCI-E 4.0 Height Extender, , if it will be needed.

It will upgrade the current X570+R9 3900X based build(Photo attached).
Not the cleanest thing in the word, and I'm not quite sensitive to dust, so my apologies in this regard
If all my GPU or RAID move to free slots plans will completely fail, the new build will look exactly the same(May-be without side+riser mounted X710-T2 NIC), and air cooler instead of AiO(Eeh... ).
If Ill succeed then:
a) GPU will move to most bottom slot, and I'll be able to install cards larger than 2.7 slots without compromising on slots in the future.
* Still, Asus announced ProArt VGA cards with 4080 finally fitting 2.5 slots. But I love my 7900 so far, AMD - Sort the drivers!, BSOD on server during drivers 23.3.2->23.4.1 update is just ... Argghhh.
b) Areca RAID will move to most bottom slot, and fans-card cooling it will move under it on case slot where is no mobo underneath, saving one PCI-E slot which currently wasted on cooler.
* May-be I'll even try to mount high static pressure fan just behind the card and it will make good cooling effect too, and Ill be able to get rid off fans-card then... Needs testing

 

[vityan666]

Thibsie

the hardware 'cos we can't afford it. Huh...

Unfortunately it's always the same for all public schools in all countries.
No budgets, no money and recently more cuts due to inflation...
I still remember everyone trying to catch the single PC in the library to surf the web and download stuff to the floppies with WinZip/WinRAR, as home Dial-Up was extremely costly let alone ISDN lines.
Or schools without computer classes at all.
Admin struggling to manage it all with NT 4 policies and other kids trying to hack it. Funny messages during lessons using the classic 'net send'.
Funny good old times.

 

[igor_kavinski]

Funny good old times.

Going into the Temporary Internet Files folder of other users and checking out the porn they had been viewing. Once, I selected all the files for deletion and accidentally pressed Enter. Hundreds of pics started opening at once. It was impossible to stop it and the PC started crawling at a snail's pace due to all the RAM being used up. Had to hard shut it down in a panic before the lab admin came over to see why my face looked ashen

 

[igor_kavinski]

I should do a costs analysis or hire my accountant to do it the next time I see to upgrade builds after this June one.

Are you sure that your media encoding workloads will scale to 192 threads? Otherwise you may need to do the encoding inside multiple VMs.

 

[A///]

Are you sure that your media encoding workloads will scale to 192 threads? Otherwise you may need to do the encoding inside multiple VMs.

if proxy encoding works it is viable, but at that 12k doesn't scratch the surface of such a setup. proxy editing like encoding still requires systems to exist outside the editing encoding rig.

 

[Markfw]

if proxy encoding works it is viable, but at that 12k doesn't scratch the surface of such a setup. proxy editing like encoding still requires systems to exist outside the editing encoding rig.

I don't have 12k in this. The CPU was a little over $5K and the motherboard/memory was $2K. The case/psu/HD/HSF was $500. So about $7500 total. ebay for the CPU was the only item not sold from a regular company, the rest was retail/new(newegg/supermicro/other). In fact the new CPU is new, I just don't have it yet.

 

[A///]

I don't have 12k in this. The CPU was a little over $5K and the motherboard/memory was $2K. The case/psu/HD/HSF was $500. So about $7500 total. ebay for the CPU was the only item not sold from a regular company, the rest was retail/new(newegg/supermicro/other). In fact the new CPU is new, I just don't have it yet.

Well I'll be. I had it in my head the 96 core was around 10K.

 

[Markfw]

Well I'll be. I had it in my head the 96 core was around 10K.

Here is the cheapest one at the moment. A retail,, not a QS. They go up from there, but you can get one for ~5k if you offer.

9654 AMD EPYC Server Processor (3.7 GHz, 96 Cores, Socket SP5) Tray -... | eBay

<p>9654 AMD EPYC Server Processor (3.7 GHz, 96 Cores, Socket SP5) Tray -...</p>

www.ebay.com

 

[A///]

Here is the cheapest one at the moment. A retail,, not a QS. They go up from there, but you can get one for ~5k if you offer.

9654 AMD EPYC Server Processor (3.7 GHz, 96 Cores, Socket SP5) Tray -... | eBay

<p>9654 AMD EPYC Server Processor (3.7 GHz, 96 Cores, Socket SP5) Tray -...</p>

www.ebay.com

Tray model. These don't come in retail box do they? I'll have to check these out when I get back into town in a few days. Thanks and cheers.

 

[Markfw]

Tray model. These don't come in retail box do they? I'll have to check these out when I get back into town in a few days. Thanks and cheers.

Mine just come secured properly. No box. But in the past were new. I will see tomorrow. The point is, not 10K, but 5k (approx)

 

[igor_kavinski]

The point is, not 10K, but 5k (approx)

That's a lot of cores for not a lot of money. Intel could counter with 96 or 128 Gracemont cores but then the performance wouldn't be the same. Wow. AMD is really on a roll.

 

[Markfw]

That's a lot of cores for not a lot of money. Intel could counter with 96 or 128 Gracemont cores but then the performance wouldn't be the same. Wow. AMD is really on a roll.

I don't know if you realize, but the 9654 has broken MANY world records, and at a very efficient power rating. Intel can not touch this CPU at present. The best does not come cheap.

 

[Soulkeeper]

I'm curious how the bios overclocking (voltages, memory, etc.) compare to the enthusiast motherboards most of us are used to ?

 

[Markfw]

I'm curious how the bios overclocking (voltages, memory, etc.) compare to the enthusiast motherboards most of us are used to ?

Most server motherboards do not allow any type of overclocking. from what I have seen so far, nothing at all for this one. Might have a setting got 320 or 400 watt. Both are in spec, but the 400 requires much better cooling.

 

[Markfw]

Its not done, but its live.

 

[jonperron09]

Its not done, but its live.
View attachment 79601

Nice! Glad to know it was just a locked CPU then.

I feel less stressed already

 

[igor_kavinski]

Markfw is a true server Pro!

Personally, my hands won't stop shaking with hardware this expensive.

I got all sweaty and nervous working on my $700 12700K PC.

 

[VirtualLarry]

192 CPU cores, Mark - beautiful!