My 7452 is doing 4.5 hours/task ! I will look at messing with the 7742 later based on the above.
My 7452 is doing 4.5 hours/task ! I will look at messing with the 7742 later based on the above.@Markfw, my dual-7452 with PPT and TDP at 180 W, "ACPI SRAT L3 Cache As NUMA Domain" = "Enabled", run 16 tasks at once per computer ( = 8 tasks at once per socket), 4 threads per task (i.e. I only use half of the logical CPUs to save a few Watts), and average duration of the latest >30 completed tasks are:
Main tasks, 3*2^n+1 form, >9,100 credits — 7.4 h
Main tasks, 3*2^n-1 form, >8,700 credits — 7.5 h
Edit,
according to my scripted tests prior to the challenge, the exact same setup except without the NUMA tweak would result in 28% longer runtimes, IOW throughput would be reduced to 78%.
Edit 2,
for reference, a 7452 has got 8 CCXs, each one with 4c/8t and 16 MB L3$.
Forget that.... New ETA is 17.5 hours.OK, if I forget the older units, the 2 new units will finish in 6 hours on the 7742. And thats with 8 threads (x8 each) running !
I have an ASRock Rack and I can not find anything close to that in bios. I guess I have to live with 17.5 hours.....I took measurements on a dual-7452. These measurements will be representative for a single-7452 as well. (The dual-7452 spends a part of the energy budget for the Infinity Fabric link between the two sockets, but it should not be much because inter-socket traffic is low in this sort of workload. Also, I increased PPT and TDP to 7452's possible maximum of 180 W in the BIOS, default is 155 W.)
I performed the measurements with a fixed workunit, in a scripted testbed. That is, the workloads consisted of multiple tasks from the same workunit running in parallel, and the same workunit being re-used in all test scenarios with different parallel task count and thread count. The consequence is that these tests are very precise, repeatable, and quick.
In contrast, observations of random workunits coming from PrimeGrid are not as conclusive, especially because there are two types of "main tasks" which have not only different durations but also different PPD, as I mentioned in #153.
Which particular tests I ran and what the results were is posted in a private section of the teamanandtech.org forum. However, I spilled the beans in #153 already.
One 7452 has got 8 CCXs, each CCX made up of 4c/8t and 16 MB L3$. Here is how I am configuring them for the challenge:
- In the BIOS, "Advanced" --> "ACPI Settings", I am switching "ACPI SRAT L3 Cache As NUMA Domain" from "Auto" to "Enabled". (That's how it is labeled in Supermicro's AMI BIOS.)
The effect of this is that the firmware will present each CCX as a NUMA node to the operating system. A NUMA aware OS like Linux will attempt to keep multithreaded processes within a NUMA node. Declaring a CCX a NUMA node is a bit of a hack as a replacement for cache-aware scheduling. The latter is more problematic than NUMA-aware scheduling, and I am not sure whether cache topology plays a role at all in current Linux scheduler decisions. (There is a related scheduler change in Linux 5.16 which I mentioned elsewhere, but this shouldn't affect all-core loads.)
In the output of the "lscpu -e" command, the NODE column will show the effect of this change. A dual-7452 system will then have the nodes 0…15 instead of nodes 0…1.
- For the first 4+ days of the 5 challenge days, I will run 8 tasks in parallel on each 7452 (that is, 16 tasks at once on a dual-7452 computer) = a 1:1 ratio of # tasks to # CCXs.
- I choose to let each task have 4 program threads.
- However, the CPU could of course give 8 hardware threads to each task, since there are 8 threads in each CCX. And indeed, using all threads would increase performance by a small percentage. It would also increase power draw but more than proportionally.
- So, since EPYC Rome isn't a power hog in the first place, you might prefer to go with 8 threads per task and spend that little bit of extra electric energy.
________
- During the last day, if I find the time, I may switch to fewer tasks at once combined with more program threads per task. This will sacrifice throughput but decrease run times. That way, the last hours of the challenge will be better filled out.
Edit: There is one thing though which I haven't considered yet at all. It's whether @cellarnoise is to be classified as impish or as admirable.
That's with 8 tasks at once.@Markfw, my dual-7452 with PPT and TDP at 180 W, "ACPI SRAT L3 Cache As NUMA Domain" = "Enabled", run 16 tasks at once per computer ( = 8 tasks at once per socket), 4 threads per task (i.e. I only use half of the logical CPUs to save a few Watts), and average duration of the latest >30 completed tasks are:
The stderr.txt says this is with 16-threaded tasks. If boinc had 100% of the CPUs, that would be 4 tasks at once. But in #172 you said it's only 3 tasks at once.My 7452 is doing 4.5 hours/task !
Supermicro has this under Advanced/ ACPI Settings, but it could be anywhere, or indeed not present at all. Related to this option, AMD's EPYC Rome workload tuning guide describes another ACPI setting in which the user could modify the "NUMA distance" of the L3$ domains. Supermicro's BIOS does not expose such a setting in the BIOS setup menu. So it's reasonable to assume that other BIOS vendors expose even fewer of AMD's tuning options.I have an ASRock Rack and I can not find anything close to that in bios. I guess I have to live with 17.5 hours.....
The only easy way I have found to do that is remove the project, and re-add. On 9 computers, what a pain. The 7452 is saving 2% for the video card. The 8 tasks at once is what happened after I added your config and rebooted. There is still 1/2 the threads available.That's with 8 tasks at once.
The stderr.txt says this is with 16-threaded tasks. Assuming you gave boinc 100% of the CPUs, that would be 4 tasks at once.
From http://www.primegrid.com/results.php?hostid=1092034:
Main tasks, 3*2^n+1 form, >9,070 credits — 4.5 h
Main tasks, 3*2^n-1 form, >8,600 credits — 4.8 h
BTW, it seems you still had quite a few tasks in your work buffer which were downloaded before the challenge started. It would have been better to abort all of these at the beginning of the challenge, then download new work. The results of work downloaded before the challenge do not count.
This is sufficient in such cases:The 8 tasks at once is what happened after I added your config and rebooted.
The computers which I have in this challenge are all headless. Except for one Xeon with 3 GPUs, but I left these GPUs idle. It's rare that I use CPUs and GPUs together, it's just too much heat if I start up everything.The 7452 is saving 2% for the video card.
As a reminder, for points in this challenge, the 321-LLR subproject needs to be selected (CPUs only, with multithreading support), work must be downloaded after Monday March 21, 03:21 UTC = today, Sunday March 20, 23:21 EDT, 20:21 PDT, [...]
It's too late to bring this up now of course, but it may help in future PrimeGrid Challenges:If you have work in the buffer which was downloaded earlier than that, just abort this work and update the project.
OK, I could not find PL1 and PL2 limits. I found something about turbo performance and disabled it. Then another setting mentioning turbo. I disabled it. I changed video cards, as I could not even see the power or menu icons, to a very tiny video card (no external power cables at all). Now its running P cores only and 78 watts !!!! Question is, now how fast will it run. Current ETA says over 6 hours. Maybe I will reenable those turbo buttons.Aren't there BIOS options for short term and long term power limit? 300+ W are insane, and I bet that the chip is only marginally faster than if it was operating at, let's say, 120 W (which would still be borderline obscene for an 8+4 core desktop CPU).
Apparently, BIOS vendors of DIY mainboards tend to make some very silly choices for their default parameters.
Those who can't see blood need to look away now; it's going to be more like this than a veggie meal... boss of SETI.Germany and vaughan of AMD Users apparently want some of that too.Looks like a souper messy fight for 4th place!