Question Genoa builders thread.

[mikegrok]

This is probably not what you are looking for, but if you need a ducted air system that moves a lot of air, it is hard to beat the replacement motor and blower for a whole home vacuum. You can probably put this on a dimmer circuit to make it quieter, but it will definitely have enough suction for whatever you may need it for. I think I have seen them used 3 times as computer fans. in a Blacksmith shop, A pottery Studio, and at a welder/exhaust repair shop. They put some furnace vent filters on one side of a wood box, and then the computer, then this motor. It is more compact than a HVAC blower, and keeps moving air even though the filter is mostly clogged. It was in the same city, and may have been setup by the same guy.

Lamb 116210-85 Motor for Vacuums and Blowers - Sales, Parts, Troubleshooting

Lamb Ametek 116210-85 motor. 110-120 Volt, 2-stage, Width 5.7, Height 7, Waterlift 97.4, CFM 99.1, AirWatts 304, Amps 7.4.

builtinvacuum.com

 

[StefanR5R]

The electric power spent in the cooling system would then exceed the power taken by a single-socket computer. So that's something you want to cool several sockets with at once.

 

[StefanR5R]

CPU: AMD Epyc 9554P (64 cores, 8 fully enabled CCDs, 360 W TDP, 320...400 W cTDP)
mainboard: unsurprisingly, Supermicro H13SSL-N
RAM: 12x 32GB DDR5-4800 reg ECC
storage: Samsung PM9A3 M.2

case: Silverstone FARA R1, front panel removed — to be replaced by a slightly wider case, as discussed below
PSU: Seasonic Focus PX 650W, platinum rated

CPU cooler: Alphacool ES Jet LGA 4677 2U, adapted to socket SP5 by means of Alphacool ES Epyc SP5/LGA6096 Mounting Kit
pump: a D5 from EKWB
radiators: a 360x120x60 mm³ and a 240x120x40 mm³ radiator from EKWB
Pump, radiators, the little reservoir, fittings and much of the tubing were previously used in a watercooled dual-GPU computer with a bit more cooling demand than this one.

radiator fans: 5x Arctic P12 PWM PST (200...1800 rpm 120mm fans), mounted in 'pull' mode for easier dust management
sort of a mainboard fan: 1x Noctua NF-A20 PWM suspended in the case to keep air moving over the mainboard, prominently blocking the view to all the interesting bits



I could have mounted the top radiator with regular bolts at the designated mounting spots of the top grille, but then it would have touched the DIMMs. Therefore I zip-tied it for now, for more lateral offset. I am going to get a wider case with more clearance between top radiator and RAM. (An alternative would be to just drill extra mounting holes into the top of the FARA R1, for more radiator offset. But I also want to improve the looks of the front.)

I have the NF-A20 running at full speed but the pump and the five P12 fans at about 2/3rds speed. If I pull them up to full speed, that adds 20 W to the system power consumption.

With the 9554P set at its default 360 W TDP (and, I think, identical PPT), and 64 instances of PrimeGrid SGS-LLR running (this is an AVX-512 heavy load of which the hot data fit into L3$), the entire system pulls 420 W at the wall. The cores are going at 3.3 GHz, and CCD temperatures are 52...55 °C. Ambient temperature is 25 °C.

$ sudo ipmitool sensor
CPU Temp         | 55.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 100.000   | na        
Inlet Temp       | na         |            | na    | na        | na        | na        | na        | na        | na        
System Temp      | 41.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 85.000    | na        
Peripheral Temp  | 38.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 85.000    | na        
CPU_VRM0 Temp    | 64.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 100.000   | na        
CPU_VRM1 Temp    | 60.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 100.000   | na        
SOC_VRM Temp     | 43.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 100.000   | na        
VDDIO_VRM Temp   | 39.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 100.000   | na        
DIMMA~F Temp     | 43.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 85.000    | na        
DIMMG~L Temp     | 39.000     | degrees C  | ok    | na        | 5.000     | na        | na        | 85.000    | na        
MB 12V           | 11.915     | Volts      | ok    | 10.571    | 10.739    | na        | na        | 12.839    | 13.091    
MB 5VCC          | 5.058      | Volts      | ok    | 4.594     | 4.636     | na        | na        | 5.564     | 5.606     
MB 3.3VCC        | 3.303      | Volts      | ok    | 3.022     | 3.069     | na        | na        | 3.654     | 3.678     
VBAT             | 0x4        | discrete   | 0x0400| na        | na        | na        | na        | na        | na        
MB 5VSB          | 5.016      | Volts      | ok    | 4.594     | 4.636     | na        | na        | 5.564     | 5.606     
MB 3.3VSB        | 3.311      | Volts      | ok    | 2.966     | 3.019     | na        | na        | 3.602     | 3.629     
MB 1.8VSB        | 1.840      | Volts      | ok    | 1.553     | 1.642     | na        | na        | 2.058     | 2.077     
BMC 2.5V         | 2.530      | Volts      | ok    | 2.392     | 2.412     | na        | na        | 2.608     | 2.627     
BMC 1.8V         | 1.807      | Volts      | ok    | 1.728     | 1.741     | na        | na        | 1.899     | 1.912     
BMC 1.2V         | 1.210      | Volts      | ok    | 1.151     | 1.161     | na        | na        | 1.259     | 1.269     
BMC 1.0V         | 1.010      | Volts      | ok    | 0.961     | 0.971     | na        | na        | 1.049     | 1.059     
CPU_VDDCR0       | 0.952      | Volts      | ok    | 0.515     | 0.530     | na        | na        | 1.576     | 1.654     
CPU_SOC          | 0.796      | Volts      | ok    | 0.577     | 0.608     | na        | na        | 1.357     | 1.420     
CPU_VDDCR1       | 0.952      | Volts      | ok    | 0.515     | 0.530     | na        | na        | 1.576     | 1.654     
CPU_VDDIO        | 1.092      | Volts      | ok    | 0.749     | 0.780     | na        | na        | 1.342     | 1.404     
FAN1             | na         |            | na    | na        | na        | na        | na        | na        | na        
FAN2             | na         |            | na    | na        | na        | na        | na        | na        | na        
FAN3             | na         |            | na    | na        | na        | na        | na        | na        | na        
FAN4             | na         |            | na    | na        | na        | na        | na        | na        | na        
FANA             | na         |            | na    | na        | na        | na        | na        | na        | na        
FANB             | na         |            | na    | na        | na        | na        | na        | na        | na        
M2SSD1 Temp      | 33.000     | degrees C  | ok    | na        | na        | na        | na        | 70.000    | na        
M2SSD2 Temp      | na         |            | na    | na        | na        | na        | na        | na        | na        
Chassis Intru    | 0x0        | discrete   | 0x0000| na        | na        | na        | na        | na        | na

RAM and VRM temperatures are lower than I expected, given the rather low air speed from the NF-A20.

 

[StefanR5R]

Without the side fan (and still without side panel), RAM/ VRM/ peripheral temperatures go up notably, but not catastrophically. Same workload and same ambient temperature as yesterday:

$ sudo ipmitool sensor | cut -d\| -f-3 | grep degrees
CPU Temp         | 54.000     | degrees C  
System Temp      | 59.000     | degrees C  
Peripheral Temp  | 52.000     | degrees C  
CPU_VRM0 Temp    | 83.000     | degrees C  
CPU_VRM1 Temp    | 78.000     | degrees C  
SOC_VRM Temp     | 58.000     | degrees C  
VDDIO_VRM Temp   | 57.000     | degrees C  
DIMMA~F Temp     | 61.000     | degrees C  
DIMMG~L Temp     | 58.000     | degrees C  
M2SSD1 Temp      | 46.000     | degrees C

Here is another photo, this time without the side fan and with a slight overexposure to compensate for my lack of a suitable light source at the computer's current location:



The PCIe slots, the spare M.2 slot, and the MCIO connectors at the right edge of the mainboard are covered with sticky tape for dust protection.

My routing of the tubes could be neater, but as mentioned, I'll be moving this into a better fitting case eventually.

If somebody wonders where the fill port and drain port of the waterloop are: The cap of the reservoir is my fill and drain port. The whole computer and the reservoir can be moved around easily enough. Besides, open loops such as this need to be filled only once (when the computer is built), perhaps slightly topped up after a couple of years, and drained only once (when the computer reached EOL after several years). [OK, this loop will likely be filled and drained twice because I plan to change the case.]

I commend Alphacool for building CPU coolers which are compatible with both Intel LGA 4677 and AMD SP5. Other aftermarket cooler makers should do the same! It's just a rather trivial matter of fitting the cold plate for both sockets, and providing the bolt hole pattern for both sockets.

One nit with my specimen of the cooler: The tube of thermal paste which came with it (Alphacool Subzero) had not a lot of content to begin with, and that was already dried up. Luckily I had enough Noctua NT-H1 left.

The little thingy which is laying on the shroud is Noctua's PWM controller. The computer will be either switched off, or will be loaded 100% on all cores; only very rarely will it run idle. Because of that, manual PWM control of fans and pump will be more practical than Supermicro's rudimentary fan control. For now, I've got just this one PWM controller for both the pump and the radiator fans. I might get another one to control pump and fans separately.

Some notes on the Supermicro H13SSL-N:

It has been mentioned earlier in this thread that his board is slightly deeper than an ATX board. Therefore you need to take a good look at the shape of the back plate of the computer case when you select a case for this board. And there is another compatibility issue: Two standard ATX mounting holes, one at center and one at top-center, are not present on the H13SSL-N, because the DIMM arrays are conflicting with these mounting hole positions. Therefore you need a computer case in which the standoffs at these two positions can be removed. Some cases have some or all standoffs riveted on, rather than screwed on. You can't use such a case unless you are willing to get a little destructive with a drill or dremel.

I don't know about you, but when I power up a newly built computer for the first time, I am always getting a bit nervous. The first power-up of this one was quite irritating in its own way: The BMC showed me that I have an Epyc 9634 84-core CPU installed! (Since the second power-up or so, the BMC correctly shows the 9554P. Also, the AMI BIOS showed the 9554P properly right when I entered it the first time.) Furthermore, the BMC did not show any sensors at all after the first power-up, only later times it did. About the only data which showed up reassuringly correctly in the BMC's web interface right from the start were the twelve DDR5 DIMMs, including their speeds, sizes, and so on.

POST takes quite a lot of time. Even more than what I am used from Supermicro H11DSi for example. And that's even though "memory training" is switched off by default. (And I will be leaving it off; no use for that for the RAM setup which I have.) I shortened the POST duration somewhat by disabling PXE boot probing.

Edit:
Idle power consumptions at the wall:

110...120 W on an idle desktop with a web browser window on a not too laden news website,
about 95 W with display-manager service shut down, i.e. at an idle Linux text console​

I haven't explored what power savings features there might be left to enable.

 

[Markfw]

Some notes on the Supermicro H13SSL-N:

It has been mentioned earlier in this thread that his board is slightly deeper than an ATX board. Therefore you need to take a good look at the shape of the back plate of the computer case when you select a case for this board. And there is another compatibility issue: Two standard ATX mounting holes, one at center and one at top-center, are not present on the H13SSL-N, because the DIMM arrays are conflicting with these mounting hole positions. Therefore you need a computer case in which the standoffs at these two positions can be removed. Some cases have some or all standoffs riveted on, rather than screwed on. You can't use such a case unless you are willing to get a little destructive with a drill or dremel.

I don't know about you, but when I power up a newly built computer for the first time, I am always getting a bit nervous. The first power-up of this one was quite irritating in its own way: The BMC showed me that I have an Epyc 9634 84-core CPU installed! (Since the second power-up or so, the BMC correctly shows the 9554P. Also, the AMI BIOS showed the 9554P properly right when I entered it the first time.) Furthermore, the BMC did not show any sensors at all after the first power-up, only later times it did. About the only data which showed up reassuringly correctly in the BMC's web interface right from the start were the twelve DDR5 DIMMs, including their speeds, sizes, and so on.

POST takes quite a lot of time. Even more than what I am used from Supermicro H11DSi for example. And that's even though "memory training" is switched off by default. (And I will be leaving it off; no use for that for the RAM setup which I have.) I shortened the POST duration somewhat by disabling PXE boot probing.

Yes, the CPU shows what the LAST full post of the motherboard was, so you have to ignore the first full cycle. Annoying. The post time for the 9000 series is like 5-10 minutes, and drives you crazy until you get used to it.

After that, they are awesome !

 

[cellarnoise]

Without the side fan (and still without side panel), RAM/ VRM/ peripheral temperatures go up notably, but not catastrophically. Same workload and same ambient temperature as yesterday:

$ sudo ipmitool sensor | cut -d\| -f-3 | grep degrees
CPU Temp         | 54.000     | degrees C 
System Temp      | 59.000     | degrees C 
Peripheral Temp  | 52.000     | degrees C 
CPU_VRM0 Temp    | 83.000     | degrees C 
CPU_VRM1 Temp    | 78.000     | degrees C 
SOC_VRM Temp     | 58.000     | degrees C 
VDDIO_VRM Temp   | 57.000     | degrees C 
DIMMA~F Temp     | 61.000     | degrees C 
DIMMG~L Temp     | 58.000     | degrees C 
M2SSD1 Temp      | 46.000     | degrees C

Here is another photo, this time without the side fan and with a slight overexposure to compensate for my lack of a suitable light source at the computer's current location:

View attachment 87156

The PCIe slots, the spare M.2 slot, and the MCIO connectors at the right edge of the mainboard are covered with sticky tape for dust protection.

My routing of the tubes could be neater, but as mentioned, I'll be moving this into a better fitting case eventually.

If somebody wonders where the fill port and drain port of the waterloop are: The cap of the reservoir is my fill and drain port. The whole computer and the reservoir can be moved around easily enough. Besides, open loops such as this need to be filled only once (when the computer is built), perhaps slightly topped up after a couple of years, and drained only once (when the computer reached EOL after several years). [OK, this loop will likely be filled and drained twice because I plan to change the case.]

I commend Alphacool for building CPU coolers which are compatible with both Intel LGA 4677 and AMD SP5. Other aftermarket cooler makers should do the same! It's just a rather trivial matter of fitting the cold plate for both sockets, and providing the bolt hole pattern for both sockets.

One nit with my specimen of the cooler: The tube of thermal paste which came with it (Alphacool Subzero) had not a lot of content to begin with, and that was already dried up. Luckily I had enough Noctua NT-H1 left.

The little thingy which is laying on the shroud is Noctua's PWM controller. The computer will be either switched off, or will be loaded 100% on all cores; only very rarely will it run idle. Because of that, manual PWM control of fans and pump will be more practical than Supermicro's rudimentary fan control. For now, I've got just this one PWM controller for both the pump and the radiator fans. I might get another one to control pump and fans separately.

Some notes on the Supermicro H13SSL-N:

It has been mentioned earlier in this thread that his board is slightly deeper than an ATX board. Therefore you need to take a good look at the shape of the back plate of the computer case when you select a case for this board. And there is another compatibility issue: Two standard ATX mounting holes, one at center and one at top-center, are not present on the H13SSL-N, because the DIMM arrays are conflicting with these mounting hole positions. Therefore you need a computer case in which the standoffs at these two positions can be removed. Some cases have some or all standoffs riveted on, rather than screwed on. You can't use such a case unless you are willing to get a little destructive with a drill or dremel.

I don't know about you, but when I power up a newly built computer for the first time, I am always getting a bit nervous. The first power-up of this one was quite irritating in its own way: The BMC showed me that I have an Epyc 9634 84-core CPU installed! (Since the second power-up or so, the BMC correctly shows the 9554P. Also, the AMI BIOS showed the 9554P properly right when I entered it the first time.) Furthermore, the BMC did not show any sensors at all after the first power-up, only later times it did. About the only data which showed up reassuringly correctly in the BMC's web interface right from the start were the twelve DDR5 DIMMs, including their speeds, sizes, and so on.

POST takes quite a lot of time. Even more than what I am used from Supermicro H11DSi for example. And that's even though "memory training" is switched off by default. (And I will be leaving it off; no use for that for the RAM setup which I have.) I shortened the POST duration somewhat by disabling PXE boot probing.

Edit:
Idle power consumptions at the wall:

110...120 W on an idle desktop with a web browser window on a not too laden news website,​

about 95 W with display-manager service shut down, i.e. at an idle Linux text console​

I haven't explored what power savings features there might be left to enable.

Reported!!!! To my brain to stop looking at this puter/pron... Must resist buying new junk for a little while longer!

Genoa Pron is so cpu-exy!

 

[vityan666]

Meantime after 30d of running stable(Finally), I've reached maintenance window to apply updates and tried to replace the 2-nd USB controller(USB 3.0 one) with the third one from Startech(USB 3.2 dual chip w/o hub) and it works as well with BIOS 1.5(I see PCI PERR's during boot though). Currently its 4-th day of uptime since new USB card change.

And while the main system is running I continue slowly(Due to lack of time) build DIY router to move the router VM out of main server into dedicated hardware. Yet to choose if I'll continue using OpnSense(Which supports the hardware, but needed to manually add Broadcom drivber into loader.conf) or try Mikrotic RouterOS on bare metal.
Machine has Ryzen 5600G, 2x16GB DDR4 UDIMM-ECC, ASRock Rack X570D4U-2L2T/BCM X570 mobo with BMC and Intel X710 BM-2 PCI-E network adapter.
This combination shall create relatively nice 10G router(While definitely not as power effective as Forti's or Mikrotiks, and even very power hungry compared to them) in a compact form factor. The Noctua's low profile I bougth, unfortunately didn't match,
because it needed its own backplate(And mobo's one seems to be sealed into the mobo), but fortunately the CPU came with AMD's stock fan and it was low-profile enougth for my MiniATX case, and case panel has air holes just in the fan area.

Markfw​

The post time for the 9000 series is like 5-10 minutes, and drives you crazy until you get used to it.

Fortunately the longer time applies to full cold boot, there it has first training step(With SuperMicro white logo).
Reboots and Shutdown+Power On sequences afterwards are 30-40% shorter, but still take time.
Relative to it, ASRock Rack mobo of X570 boots fast like any desktop mobo, but had very tricky DIMM slots - needed some self-training to get used to how to push the module near the 'closed/fixed' side, otherwise module didn't go all the way down, and even looking from side
didn't allow to clearly see it. I prefer SuperMicro's approach with two-sided manual latches(classic), but ASRocks BMC web interface is much better - Long session without auto-logouts every few minutes, Active Directory and SMTP integration, Semi-Automatic and fully manual Fan controls(In addition to control by zones), nice and more user friendly KVM, dedicated BMC interface used by default, instead of the inconvenient 'shared' thingy and more... On the other side Dr. Debug is less convenient, but can bee seen directly on the mobo without display/kvm.
ASRock Rack recently announced several new mobos for Bergamo and Siena, making older 'preliminary' boards finally released(Such as GENOAD8X-2T/BCM).

 

[vityan666]

Markfw​

Looks like I was right. Both HEDT and PRO TR 7000 versions use the smaller SP6 like socket(TR5). And our SP5 remains outsider...
Siena owners will be happy...

Noctua Announces CPU Coolers for AMD's New Threadripper and EPYC processors

Noctua today announced its new NH-U14S TR5-SP6 and NH-D9 TR5-SP6 4U CPU coolers for AMD's TR5 (sTR5/sTRX5/sWRX9) and SP6 sockets for the just-released 7000 series Threadripper HEDT (High-End Desktop) and Threadripper PRO workstation processors as well as 8004 series EPYC server CPUs. While the...

www.techpowerup.com

 

[StefanR5R]

ASRocks BMC web interface is much better - Long session without auto-logouts every few minutes, Active Directory and SMTP integration, Semi-Automatic and fully manual Fan controls(In addition to control by zones), nice and more user friendly KVM, dedicated BMC interface used by default, instead of the inconvenient 'shared' thingy and more...

Apparently, auto-logout can be reconfigured and even disabled in Supermicro's BMC web interface: Configuration -> BMC Settings -> Web Session -> Session Timeout Value
"Please enter an integer value from 0-30 for minutes. By default, timeout value is set to 30. Enter 0 to set to 'never timeout'."

One other inconvenience: When I log into the older web interface of Supermicro H11DSi's BMC, Firefox automatically fills in the username and password in the login screen (if I store these in Firefox). It doesn't do so in the login screen of H13SSL, there I have to enter username and password of the IPMI admin at every login. Am I doing something wrong/ is there a fix?

 

[vityan666]

StefanR5R​

Configuration -> BMC Settings -> Web Session -> Session Timeout Value

I know about this option. I have it set to 30 and it drops me out in less than 5 minutes(SMicro software bugs ), didn't tried 0 however

It doesn't do so in the login screen of H13SSL, there I have to enter username and password of the IPMI admin at every login. Am I doing something wrong/ is there a fix?

You are doing everything right. I've looked at the password input element handling code.

function turnOnPwdStyle (o)
{
   ...

    else if (src.indexOf("open") > 0 )
    {
        $("#"+o.id).attr("type", "password");
        $("#"+o.id).attr("autocomplete", "new-password");

Objects having autocomplete attribute set to 'new-password' will not be auto-filled by the modern browsers - It's purpose is only to aid in generation of new password.
I would consider it a bug(One on of zillion... ).

 

[StefanR5R]

I am giving the VRMs of the system pictured in #153 a workout now.
– PPT and TDP upped to 400 W
– vector-heavy cache-friendly work on all 128 threads
– case is open as on the photo, 25 °C ambient
– pump and radiator fans at 55%, side fan at 100%
– power draw on the wall: 484 W

$ sudo ipmitool sensor | cut -d\| -f-3 | grep degrees
CPU Temp         | 58.000     | degrees C  
System Temp      | 44.000     | degrees C  
Peripheral Temp  | 39.000     | degrees C  
CPU_VRM0 Temp    | 70.000     | degrees C  
CPU_VRM1 Temp    | 67.000     | degrees C  
SOC_VRM Temp     | 43.000     | degrees C  
VDDIO_VRM Temp   | 41.000     | degrees C  
DIMMA~F Temp     | 44.000     | degrees C  
DIMMG~L Temp     | 43.000     | degrees C  
M2SSD1 Temp      | 30.000     | degrees C

That is, the CPU is cooled with ease, but the VRM coolers sure want their airflow.

While I was purchasing the parts of this computer, I had been looking around for VRM watercoolers. While there are some (listed as) available with a fairly generic cold plate geometry, they wouldn't be small enough to fit between the capacitors and coils which are located adjacent to the VRMs. One would either have to mill dedicated coldplates, or put a metallic spacer (copper, or more readily available brass maybe) between VRMs and the generic cold plate and make do with the double thermal interface.

 

[Trotador22]

<70ºC is still a great temperature for the CPU VRMs, what is the CPU clock with 400 W cTDP?, your 9554P is retail, right? Thanks!

 

[StefanR5R]

<70ºC is still a great temperature for the CPU VRMs,

Indeed, the VRM temps are OK, but the large side fan is quite audible at full speed (even in the mix with the pump and the other fans). I do admit that the little heatsinks which sit on the CPU VRMs and RAM/auxiliary VRMs are designed for even faster air flow than I am providing by this side fan and without air guides of any sort. So these temperatures aren't bad; agreed. There is certainly some lateral heat transfer going on through the mainboard, which sits opposite to the side fan and is also indirectly coupled with the CPU's watercooler.

what is the CPU clock with 400 W cTDP?

The workload with which I took these temps and power draw is once more a recreational one, related to prime number search as a means to brute-force falsification of a Sierpiński numbers conjecture. The 9554 has got 8 CCXs, and so I launched 8 instances of the prime search at once and tied each task to logical CPUs of a CCX.

• 8 threads per task = 1 thread per core gave ~465 W total system power consumption IIRC, and 3.65 GHz average core clock, which is already quite near the 9554's F_max of 3.75 GHz.
• 16 threads per task = 1 program thread per SMT thread gave the mentioned 484 W total consumption, and 3.30 GHz average core clock.

your 9554P is retail, right?

Yes, it's a retail CPU. On a side note, Genoa retail prices are all over the place and in flux (but most of the time, pretty bad in Europe), and the same as with the older EPYC series is already happening with the Genoa series: The non-P models ( = xGMI enabled for dual socket) are sometimes better priced than the equivalent P models ( = single socket only).

 

[StefanR5R]

Some BIOS options don't work as expected:

• At least with TDP = PPT = 400W (cTDP_up of this OPN), "Advanced -> NB Configuration -> Determinism Control" = [Auto] / [Manual], and in case of [Manual], "Determinism Enable" = [Disable Performance Determinism] / [Enable Performance Determinism] does not make a measurable difference in performance.
  Possible PEBCAKs:
  • I tested these with one of the earlier mentioned workloads which had the cores clocking at 3.65 GHz on average = 97% of OPN F_max. Perhaps I'll try the heavier workload or/and at a lower TDP and PPT again.
  • I performed only warm boots between changes of these BIOS options, not power cycles. (With EPYC Rome, this stuff works after warm reboot.)

• "Advanced -> CPU Configuration -> AVX512" = [Auto] / [Disabled] / [Enabled] does not change the CPU capability flags at all. I.e. even if "AVX512" = [Disabled], all of the AVX512 related flags are still there, and software uses AVX 512, and does not crash.
  This time I also tried a power cycle with AVX512 [Disabled], to no avail.

The BIOS screen says "Version 2.22.1285 Copyright (C) 2023 AMI" in the footer, and "BIOS Version 1.1a", "Build Date 02/14/2023", "CPLD Version F5.0E.13" on the main tab. The Advanced -> CPU Configuration -> CPU1 Information screen shows "Microcode Patch Level A101111"

Right now, Supermicro's public download page offers BIOS version 1.5. Its release notes (PDF) make no obvious mention of any of the above issues.

If I wanted to update the BIOS, I have to update the BMC firmware too. I'm on the fence whether or not I'll do.

 

[igor_kavinski]

Anyone using this 400W cooler? https://www.amazon.com/Heatsinks-SP5-Pre-Applied-Thermal-Stacked/dp/B0C9JRG444

 

[StefanR5R]

The size of the cooler appears to be the same as Noctua NH-U12S. If fin density and build quality (and thereby thermal contact from cold plate to heat pipes to fins) is comparable with Noctua standards, then this should be able to cool a 400 W Epyc decently with fans at full speed and the corresponding air flow rate through the computer case. (Edit: …and intake air at ambient temperature of course, not preheated by a GPU.) — Of course you don't have a noise level of 29 dB anymore then. That's the noise level at idle at best.

Here is a test of the 140 mm/ 120 mm/ 92 mm single tower coolers from Noctua on a 500 W LGA4677 Xeon:
https://www.hardwareluxx.de/index.p...ei-500-w-noctua-kühler-im-vergleichstest.html (in German)
Edit 2: Sorry, I just now realized that this test is borderline worthless. They don't mention at all whether they tested on a table or in a case, and if the latter, how it was ventilated.

 

[Markfw]

Anyone using this 400W cooler? https://www.amazon.com/Heatsinks-SP5-Pre-Applied-Thermal-Stacked/dp/B0C9JRG444

I believe I have 2 of those. They keep the 9554 and the 9654 between 87 and 94 when under full load. Right now the 9654 shows 43c, but its idle. WCG is not sending me any tasks.

Here it is on the 9654 working

http://www.portvapes.co.uk/?id=Latest-exam-1Z0-876-Dumps&exid=attachments/20230602_204629-1-jpg.81309/

And what it looks like from the BMC interface.

 

[Trotador22]

The same manufacturer has two models with better performances (supposedly):

AMD SP5 M98 AMD SP5 series AMD CPU series Product Center Leopard radiator_CPU radiator_server radiator_Dongguan radiator manufacturer-Dongguan Shuyun Hardwar

www.coolserver.com.cn

AMD SP5 M99 AMD SP5 series AMD CPU series Product Center Leopard radiator_CPU radiator_server radiator_Dongguan radiator manufacturer-Dongguan Shuyun Hardwar

www.coolserver.com.cn

 

[Markfw]

The same manufacturer has two models with better performances (supposedly):

AMD SP5 M98 AMD SP5 series AMD CPU series Product Center Leopard radiator_CPU radiator_server radiator_Dongguan radiator manufacturer-Dongguan Shuyun Hardwar

www.coolserver.com.cn

AMD SP5 M99 AMD SP5 series AMD CPU series Product Center Leopard radiator_CPU radiator_server radiator_Dongguan radiator manufacturer-Dongguan Shuyun Hardwar

www.coolserver.com.cn

If you like 10,000 rpm jet engines in your house maybe

Edit: when I click on that link the header is the bad one. The good ones are below. I see now his link whats M98 and M99 in the link.

 

[Markfw]

Ah, bad website, this is the one you DON't want. (or one thats looks like it)

Also, clarification, there is M97,M98 and M99 with 400,430,450 watt respectively. I have the 400 watt, I want the 450 watt. The one in the post above is the M99, 450 watt,

 

[StefanR5R]

I don't think you really want a 4U cooler. What you want, but nobody is making, is a 5U cooler for SP5.

(And what I for one would be interested in would be a 5U cooler for SP5 which is twice the size of the available 5U coolers for SP3 and SP6.)

 

[igor_kavinski]

(And what I for one would be interested in would be a 5U cooler for SP5 which is twice the size of the available 5U coolers for SP3 and SP6.)

Would stacking a 4U cooler on top of the existing 4U cooler work? I realize that the server would need to be open air in that case. Or maybe enclosed in a small tent?

 

[StefanR5R]

@igor_kavinski, imagine the coolers pictured in post #48 joined to a single one. This would do for living room compatible air cooling of Epyc 9004.

 

[Markfw]

OK, Genoa number 5 build starting. At least the parts are ordered and in the process of being shipped or are shipped.

9554 64 core Genoa
heatsink in http://www.portvapes.co.uk/?id=Latest-exam-1Z0-876-Dumps&exid=threads/genoa-builders-thread.2611963/post-41110444
12 16 gig 4800 DDR5

Supermicro MBD-H13SSL-N-O R​

and various other parts. Should be up by the 10th=15th of this month.

 

[Iketh]

Anyone have a link to purchase the Cooler Server 4U-M99?