Info 11th BOINC Pentathlon 2020

[Markfw]

And while posting and catching up on other posts, the usage went up to 17, now 18 gig and still rising. I just stopped a major disaster from happening, 3 crashed box (or 4)

 

[Endgame124]

All done... The three threadrippers I have sold had all that extra memory. Now all 3 3900x are at 32 gig and the Xeon !

And the 2 I did that were not locked up were using 14.5 gig of 16, so any minute they would have been a problem.

I have WAY too much hardware. I have 3 functional boxes not fired up, as I don't have the power to run them, including a 14 core 28 thread Xeon.

Edit, and I changed back the box that I had to switch to WCG, so now all 3 3900x's are on Rosetta, the 3950x, all Xeons and EPYC's, and a 2970wx and a 2990wx.

Sell the older stuff (though It looks like your stuff is all pretty new and get 4000 series Ryzen gear in a few months?)

 

[Markfw]

Sell the older stuff (though It looks like your stuff is all pretty new and get 4000 series Ryzen gear in a few months?)

I just sold 3 1950x's recently, and the 14 core Xeons, I can't even get $200 for, might as well keep. Thr 950 I7 I have I will keep for old games, and the 4790 keep for old work thinks I might want to look up. So nothing saleable left.

 

[Endgame124]

I just sold 3 1950x's recently, and the 14 core Xeons, I can't even get $200 for, might as well keep. Thr 950 I7 I have I will keep for old games, and the 4790 keep for old work thinks I might want to look up. So nothing saleable left.

Sounds like the same reasons I still have my core 2quad and APUs.

 

[StefanR5R]

you should be able to configure zram on Linux systems to create a Dynamically sized, compressed swap Partition in ram,

I think I'll go look up MemDoubler on Aminet. ;-)

I have WAY too much hardware. I have 3 functional boxes not fired up, as I don't have the power to run them,

With a bit of balancing, I am still able to run all my computers in a single room on two circuits (16 A breakers, 230 V). But to be safer, I moved a triple-GPU computer into the kitchen on another circuit.

A year ago shortly after I moved here, I fixed a wall outlet — which doubled as junction box — which got hot and smelly when I had high load on the respective circuit. I still have several outlets which get warm now (but not actually hot), and should look into that after the Pentathlon.

 

[Endgame124]

I think I'll go look up MemDoubler on Aminet. ;-)

lol! In all seriousness, zram does work. my pi 3B+ is functioning like it has 3.5gb ram, even though it only has 1gb.

If you have spare ram on hand and your system isn’t maxed, that is of course the better solution.

 

[Rattledagger]

Looks like too much or too complicated to digest? Ignore the details. Simply run Rosetta@home for TeAm AnandTech for the duration of the Pentathlon. :-)
Thanks!

Ah, yes, I'll definitely use this suggestion instead of the confusing and overly complicated stick-throwing contest.

Hey! Good to see you mate!
How's things?

Hello, Assimilator1.
A little bored with corona-lockdowns, but apart for this thinks are ok.
Oh, and I've still not learned correct usage of then and when and similar t/w-combinations.

 

[StefanR5R]

Marathon — Rosetta@home
I took a snapshot of 256 currently running tasks. All of them are rosetta_4.20_x86_64-pc-linux-gnu tasks with circa 7 hours run time so far.



minimum: 381 MBytes — average: 856 MBytes – maximum: 1,483 MBytes


Javelin Throw — NumberFields@home

Looks like too much or too complicated to digest? Ignore the details. Simply run Rosetta@home for TeAm AnandTech for the duration of the Pentathlon. :-)
Thanks!

Ah, yes, I'll definitely use this suggestion instead of the confusing and overly complicated stick-throwing contest.

With CreditNew used at NumberFields, it may turn out more like a spaghetti throwing contest. ;-)

 

[biodoc]

They released Rosetta v4.20 for linux and RAM usage definitely went North! The previous version (4.15) was fine on my 3700X with 1 GB per thread but am now seeing "waiting for memory" and swap file usage so I cut back to 14 threads.

 

[Endgame124]

They released Rosetta v4.20 for linux and RAM usage definitely went North! The previous version (4.15) was fine on my 3700X with 1 GB per thread but am now seeing "waiting for memory" and swap file usage so I cut back to 14 threads.

Is this how we get ahead of the other teams then? Have everyone who can’t buy more ram configure zram and keep their Linux systems at max processes?

Setup zram:
#install
sudo apt-get install zram-tools

#configure
sudo vi /etc/default/zramswap
#update max percentage of system ram to allow to be created as in memory swap and compressed with LZO:
PERCENTAGE=75

#restart the the zram swap service:
sudo zramswap stop
sudo zramswap start

#check zram status with
zramctl

#example zramctl output from raspberry pi 3B+ running 3 rosetta processes with a zram configuration of 250%

pi@Pumpkin:~ $ zramctl
NAME ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram3 lzo 483.5M 323.2M 144M 150.6M 4 [SWAP]
/dev/zram2 lzo 483.5M 324.7M 145.7M 152.3M 4 [SWAP]
/dev/zram1 lzo 483.5M 329.4M 148.4M 155M 4 [SWAP]
/dev/zram0 lzo 483.5M 324.5M 144M 150.7M 4 [SWAP]

 

[Markfw]

I think I'll go look up MemDoubler on Aminet. ;-)


With a bit of balancing, I am still able to run all my computers in a single room on two circuits (16 A breakers, 230 V). But to be safer, I moved a triple-GPU computer into the kitchen on another circuit.

A year ago shortly after I moved here, I fixed a wall outlet — which doubled as junction box — which got hot and smelly when I had high load on the respective circuit. I still have several outlets which get warm now (but not actually hot), and should look into that after the Pentathlon.

I have 15 amp breakers on 110/120v. That 230 volt doubles your capacity., so 4 of my circuits. Well, guess what ? Thats the 4 circuits I use.

 

[StefanR5R]

@Endgame124, and which memory settings in BOINC go with it? Allowing high RAM percentage and high swap percentage?

BTW, on computers with Intel HyperThreading, the drop in Rosetta throughput when not using all processor threads should be rather moderate. The loss is probably higher if AMD Zen/ Zen2 SMT cannot be fully used. (Luckily for me, my Zen2 CPUs have enough RAM attached to use all threads at Rosetta.)

Also, how much is the gain really, compared between a computer which uses fewer processor threads, and a computer which uses more or all threads but has Rosetta performing accesses to compressed memory? Edit, my guess is that accesses to compressed pages may be infrequent though. Rosetta is not excessively sensitive to memory speed from what I remember, despite its large memory footprint.

 

[biodoc]

It looks like these tasks are taking more than 1 GB of RAM:

Junior_HalfRoid_design5_COVID-19_SAVE_ALL_OUT_IGNORE_THE_REST_3ii3yt0o_924662_15

I do have another couple of 8 GB DDR3200 RAM sticks but they are CL16. The RAM in the box is 3200 but CL14. Normally I would jump off the roof before mixing vendor/timings RAM in the same rig. I guess I could drop them in and do a short memtest to see if it's ok.

 

[Markfw]

It looks like these tasks are taking more than 1 GB of RAM:

Junior_HalfRoid_design5_COVID-19_SAVE_ALL_OUT_IGNORE_THE_REST_3ii3yt0o_924662_15

I do have another couple of 8 GB DDR3200 RAM sticks but they are CL16. The RAM in the box is 3200 but CL14. Normally I would jump off the roof before mixing vendor/timings RAM in the same rig. I guess I could drop them in and do a short memtest to see if it's ok.

On the box I am typing, I have 2 3600 cl16 and 2 4233 cl19 running 3200 cl16. better more ram than faster/better cl ratings.

 

[Assimilator1]

I was just about to ask about the Rosetta url, I was wondering whether tasks would have to be finished, now I know .

Rosetta states 'change when convenient', there isn't a convenient time!

 

[Endgame124]

They released Rosetta v4.20 for linux and RAM usage definitely went North! The previous version (4.15) was fine on my 3700X with 1 GB per thread but am now seeing "waiting for memory" and swap file usage so I cut back to 14 threads.

Is this how we get ahead of the other teams then? Have everyone who can’t buy more ram configure zram and keep their Linux systems at max processes?

@Endgame124, and which memory settings in BOINC go with it? Allowing high RAM percentage and high swap percentage?

good question:

Assuming 75% zram swap device:

#Edit the Boinc global preferences to use more than 100% of system memory:
sudo vi /etc/boinc-client/global_prefs_override.xml

#update xml to include:
<global_preferences>
<max_ncpus_pct>100.000000</max_ncpus_pct>
<vm_max_used_pct>90.000000</vm_max_used_pct>
<ram_max_used_busy_pct>175.000000</ram_max_used_busy_pct>
<ram_max_used_idle_pct>175.000000</ram_max_used_idle_pct>
<cpu_usage_limit>100.000000</cpu_usage_limit>
</global_preferences>

BTW, on computers with Intel HyperThreading, the drop in Rosetta throughput when not using all processor threads should be rather moderate. The loss is probably higher if AMD Zen/ Zen2 SMT cannot be fully used. (Luckily for me, my Zen2 CPUs have enough RAM attached to use all threads at Rosetta.)

Also, how much is the gain really, compared between a computer which uses fewer processor threads, and a computer which uses more or all threads but has Rosetta performing accesses to compressed memory? Edit, my guess is that accesses to compressed pages may be infrequent though. Rosetta is not excessively sensitive to memory speed from what I remember, despite its large memory footprint.

If I’m reading top correctly, kswapd has been running 158 minutes over the last 6 days on my pi. It seems to get busy every 30 minutes or so for 10 seconds, then goes idle, unless I pickup one of the really ram heavy WUs, then it is busy more often.

It’s possible I would see a higher average credit if I lowered it to 2 parallel WUs, but I figured I would let it run for a while first before I go and start changing things.

 

[StefanR5R]

The third contest,

City Run — Universe@home (May 7 – May 11 inclusive = 5 days)​

has been announced 1½ hours ago.
I added details post #1.

I also added a calendar. Dates in this calendar refer to UTC; Americans need to mentally shift the dates a little. Australians and Indonesians too, but the other way. %-)

 

[Markfw]

They released Rosetta v4.20 for linux and RAM usage definitely went North! The previous version (4.15) was fine on my 3700X with 1 GB per thread but am now seeing "waiting for memory" and swap file usage so I cut back to 14 threads.

How do you get 4.2 ? mine are all on 4.15 I think

 

[StefanR5R]

The v4.20 application version was added on May 1, ~23:00 UTC (~16:00 PDT).
I guess tasks for it were added soon after this.
Edit, tasks for it were emitted immediately when this application was added. everyone who fetches new work since then gets the v4.20 application automatically.

 

[Markfw]

my 2990wx boxes only have 64 gig ram, and one of them is running 50 gig used. running 4.15. What do you think it will use under 4.2 ? more than 62 ?

 

[Assimilator1]

All my new tasks are v4.2, currently on 4.15 with 10 threads it's using about 3.9 GB with a mix of tasks taking 400 - 1200 MB.
With 32GB for 'just' 10 threads hopefully I'll be ok! , the 1st 4.2 task will start in about 4 hrs, actually 6 of them will all start close after each other.

 

[Markfw]

All my new tasks are v4.2, currently on 4.15 with 10 threads it's using about 3.9 GB with a mix of tasks taking 400 - 1200 MB.
With 32GB for 'just' 10 threads hopefully I'll be ok! , the 1st 4.2 task will start in about 4 hrs, actually 6 of them will all start close after each other.

Well, I won't see 4.2 until Tuesday, as I am bunkering.....

 

[StefanR5R]

my 2990wx boxes only have 64 gig ram, and one of them is running 50 gig used. running 4.15. What do you think it will use under 4.2 ? more than 62 ?

Earlier this week, when v4.15 was current, I ran Rosetta only on 4 cores, therefore I don't have much data. But I recall seeing about the same pattern of RAM requirement as I got today from my 256 tasks sample in post #58.

 

[StefanR5R]

Note,
if you have still v4.15 tasks on your computers, check their deadline date. It needs to be after Monday May 4, 20:00 EDT/ 17:00 PDT if it is meant to be held for the beginning of the Pentathlon. Preferably with a bit of leeway after these times, because the Rosetta server may run into difficulties to respond to all requests when the Pentathlon hell breaks loose.

 

[Markfw]

Earlier this week, when v4.15 was current, I ran Rosetta only on 4 cores, therefore I don't have much data. But I recall seeing about the same pattern of RAM requirement as I got today from my 256 tasks sample in post #58.

I see the graph, but not sure what its telling me. I guess I will have to see Tuesday the average for 12 core, 16 core, 24 cores and 32 cores with SMT on.

At least I only have one EPYC box that will run short, the 7742 only has 110 gig for 128 threads.