1000-1200 clients looking for work! :D

[babbage]

I have got between 1000 and 1200 pc's available for distributed computing

Currently I am deploying DNETC.exe for OGR crunching.

However before I go to far into this (currently at about 200 deployed) I am looking around at other projects because to be honest the OGR thing is getting a bit dull and boring.

So what I want is a client that does the following.

These are mandatory requirements not for negotiation, humanitarian / social / scientific worthiness is not a big issue for me.


1 stable client...no crashes....ever!

2 ease of deployment ... currently using rscc.exe to deploy dnetc clients would like similar usability in new project
basically a tool or app that allows me to feed list of pc's and amend the ini file etc, checks the status
checkout rscc.exe for dnetc.exe to see what i mean

3 Proxy servers i.e personal proxy in dnetc case

4 minimal wu sending and receiving ... DPAD is the ultimate in this regard

5 wu/results sent received by http port 80 ... I will not open special ports in the firewall

6 install as a service in the background

7 ability to halt/suspend client when particular application is running i.e winword etc



Desirables


good monitoring tools

good statistical tools

team spirit camraderie

So any ideas ??????

The first seven items are non negotiable, I have ultimate responsibility for the pc's and will not compromise them......sorry


Regards

Babbage

 

[Adul]

wow!!

get on the ball guys!!

 

[aberant]

well on first look i'd have to say Seti, its remarkably stable (i've had a machine run non stop for just over a year with no crashes) and offers all the facillities you mentioned. The only thing about going through port 80 is you'd have to be rerouted through a team members q that receives on port 80 as berkley doesn't accept on 80, but i'm sure that's doable BUT, seti is due to end at some point this year, possibly sooner rather than later. If its a big deal to remove and install a different client (which i would imagine would be the case with that many machines) then perhaps it'd be better to look at another more long term project. I'm sure someone from FaH or the likes will be in contact soon Perhaps run seti till it ends ? Would sure be appreciated

PS Welcome !

 

[babbage]

Yup the port 80 is an issue, currently using proxy80.teamanandtech.com to solve that for dnet.

If deployment tools existed for seti that allowed a rapid automated deployment usind prebuilt hostname lists then I wouldn't have a problem with the finish time

Regards

Babbage

 

[Adul]

I do believe there are tools to deploy seti client accross large networks easily. Also you can run it as a service or even use programs like sethide to run it hidden to cache a few WU.

specs of the 1200 PC herd

 

[Confused]

w....o....w.....!

I'm sure the Seti people will jump on this opportunity. I wish I knew how to set up some other DC clients remotely nice and easily...but alas, I don't

Erm, pay the plane ticket there, and i'll go round installing it on them!!


Garry

 

[babbage]

Many and varied

roughly as follows

200 off P4 2.40
700 off PIII 1.4
200 off PIII 1.0

most have 128mb memory

43 have 512

Regards

Babbage

 

[Freewolf]

I would also have to say seti. Where Smoke and crazee when you need them

 

[Rattledagger]

5; Seti@home uses standard HTTP, so AFAIK this should be port 80.
3; SetiQueue on the other hand is normally using 5517 or 5512 but this is only internally on your LAN so if not every box has a firewall running your own setiqueue will still talk to Berkeley by standard http. Internally there shouldn't be a problem either, since setiqueue uses the port you chooses.

2, 6; As for deployment of seti, someone around here programmed something called flexi-installer that basically needs your user_info.sah & a list of machines to install to and all of these machines should very fast have installed seti as a service. Someone else will probably soon chime in with more info on this.
7; If using seti-hide it's apparently possible to list applications that should pause seti... Maybe possible to incorporate this into the flexi-installer...

4; Seti crunches around 4wu/day per GHz. Wu is 350KB and results around 5KB.

1; As for stability, of the clients I've tried seti@home is the most stable. Other is: genome@home (project ended), folding@home, distributed folding, UD Think (or whatever they're called now...), climateprediction.

 

[amdxborg]

Hey babbage, welcome to the TeAm AnandTech DC Forums!

I suggest DPAD, and since I saw you mention it, you must know a thing or two about it. We at DPAD would love to have you as a TeAmMate. If you've got any questions about it, please ask!

If not DPAD, then any other AnandTech TeAm that really needs help!

 

[Adul]

man 1200 PC to go towards Dpad :Q

 

[amdxborg]

Originally posted by: Adul
man 1200 PC to go towards Dpad :Q

We do after all have the best looking babes and coldest beer in DC...

 

[babbage]

thougt dpad required ftp access and individual connection to the internet per pc i.e. no proxy????

regards

Babbage

 

[Overkiller]

Let me toss some Information out there:

I am a whole-hearted supporter of DPAD (i made the guide w/ ur guys input, recruited, etc etc) but DPAD is still BETA and clients do change OFTEN & there ARE problems w/ the client. After the client comes out of beta i would say yes to such a large-scale under-taking

SETI:

OK, yeah we are in a race and we need to win. I'm 100% in support of this but A) it ends w/in 6 months & B) the data has been crucnhed already 4 times over. It's just for stats

I would look into the F@H team. They are 24 and this addition could easily put them in the top 10 i believe. Folding at home is a great project that seems to not have too many "large" users on it

Also there is FAD, UD, DF, D20l, & more than i know.

But as a die-hard supporter of the DPAD team I would say don't do it (please dont hate me guys!!!!!!!!!!) simply b/c the client is still BETA and it does have some issues. its in version 4.34 right now. version 5.xx will be the full version and that should be around summer we hope if not in the fall.

So my 2nd choice would be F@h. my .002 cents. please dont flame! i was honest

 

[babbage]

Appreciate the honesty, thanks

I will be deploying a 100 pc test of whichever project seems best fit, to check for hidden issues.

Leave it run for a week 24/7 and re-evaluate.

So recommending an unstable/unsuitable project is just wasting my time, your (plural) time and everybodies time.


I dont mind dedicating some time to this at the moment but obviously I have to earn my crust as well.


Haven't looked hard at the f@h but will now to see what's what


Regards

Babbage

PS Dont mean to sound arrogant or dictorial about this, but its my ar*e on the line if this falls over and crashes 90% of the network

So forgive the "It must be this attitude"

 

[MournSanity]

I'm a DPAD whore so you know what I am going to say. Although 1000 PCs will help DPAD tremendously.


EDIT: Yea DPAD is still in beta. Maybe you can help when the final version is released?

 

[TofBnT]

BofBnT should be around soon to describe the flex installer for you, even I can use it and I am a computer idiot.

 

[wischeez]

Originally posted by: Overkiller
Let me toss some Information out there:

I am a whole-hearted supporter of DPAD (i made the guide w/ ur guys input, recruited, etc etc) but DPAD is still BETA and clients do change OFTEN & there ARE problems w/ the client. After the client comes out of beta i would say yes to such a large-scale under-taking

SETI:

OK, yeah we are in a race and we need to win. I'm 100% in support of this but A) it ends w/in 6 months & B) the data has been crucnhed already 4 times over. It's just for stats

I would look into the F@H team. They are 24 and this addition could easily put them in the top 10 i believe. Folding at home is a great project that seems to not have too many "large" users on it

Also there is FAD, UD, DF, D20l, & more than i know.

But as a die-hard supporter of the DPAD team I would say don't do it (please dont hate me guys!!!!!!!!!!) simply b/c the client is still BETA and it does have some issues. its in version 4.34 right now. version 5.xx will be the full version and that should be around summer we hope if not in the fall.

So my 2nd choice would be F@h. my .002 cents. please dont flame! i was honest

A) it ends w/in 6 months & B) the data has been crunched already 4 times over.It's just for stats

To answer you, we don't know WHEN S@H1 will end( no definite date set) and I am now crunching WU's from November of 2003 (they have NOT been crunched before.....Just so you know....

 

[MechxWarrior]

Is the DPAD client really that buggy though? At the least a few PCs could help a bit

As for FTP...It does send through FTP but as for ports I dont know.

 

[MournSanity]

Originally posted by: MechxWarrior
Is the DPAD client really that buggy though? At the least a few PCs could help a bit

As for FTP...It does send through FTP but as for ports I dont know.

It only crashed once for me, and that was the first time I used it. It hasn't crashed ever since (2 weeks), so the first crash was probably something I did. It doesn't seem buggy at all to me.

 

[Overkiller]

well also, dpad requires FTP. which i don't think he wants?

Lastly, i have read many times in these forums and elsewhere that the data has been crunched 4x over (in seti) at least and that no new is being transmitted. my apologies if i have read incorrectly.

 

[InlineFive]

D2OL!

 

[Smoke]

babbage,

I have asked someone to help you with this. He is not online at the moment.

 

[Allio]

I figure I should do the pimping for SoB, as nobody else seems to have done so

Originally posted by: babbage
These are mandatory requirements not for negotiation, humanitarian / social / scientific worthiness is not a big issue for me.


1 stable client...no crashes....ever!

We currently have very stable clients for most OSes. If in the unlikely event that something did go wrong, it would stay invisible and fix itself on the next reboot.

2 ease of deployment ... currently using rscc.exe to deploy dnetc clients would like similar usability in new project
basically a tool or app that allows me to feed list of pc's and amend the ini file etc, checks the status
checkout rscc.exe for dnetc.exe to see what i mean

I'm not sure about any methods of mass-installation... though I seem to remember the ORU guys mentioning something about it. As it stands, it only takes about 15 seconds to have the client installed and running as a service on any particular PC.

3 Proxy servers i.e personal proxy in dnetc case

Our very own Ken_g6 has written a fully featured proxy/queueing applet, if you need it.

4 minimal wu sending and receiving ... DPAD is the ultimate in this regard

SoB might be even better... it's only a few bytes a day per client. In fact, it's about 15 characters per 3 days of work

5 wu/results sent received by http port 80 ... I will not open special ports in the firewall

Fully supported, there is an alternate DNS address for the main server which listens on port 80 for these situations.

6 install as a service in the background

Comes with a service installer. One click of a batch file and it will run invisibly as a service.

7 ability to halt/suspend client when particular application is running i.e winword etc

Unfortunately, not that I know of. It does run at absolute true idle by default, though, and won't disturb even the pickiest programmes (for example, Outlook and Soulseek)

Desirables


good monitoring tools

The 'pending test' page on the website tells you each computer's progress, speed, and when it last sent in work.

good statistical tools

Very verbose real-time stats with pretty graphs

team spirit camraderie

We have a whole bunch of it, especially when our arch-enemy Team Prime Rib is on the prowl.

One thing worth mentioning is that we're currently not under any danger of losing first place, so mass-deployment is probably unneccessary! It would also make enormous splashes in what is essentially a small project. But in future, you might want to think about it

Edit: as far as I know, D2OL, SETI and DPAD are the teams needing the most help right now, so I'd drop those clients on, myself. For the time being, anyway

Good luck

 

[ProviaFan]

Well, I'd have to admit that SETI would probably fit your requirements best, though I'd love to see F@H chosen instead (the F@H TeAm could really use some help to avoid being knocked down 6 more places from 24th in the near future). However, I have no idea how in the world it would be possible to do an automated service install with F@H. However, someone more knowledgeable than I can probably help you with that if you're interested.