Mika's back!...Open the floodgates!!!!

[Engineer]

Great to have you back, We've really, really, really missed you!!!!!!

Open it up Clueless and let it flow .....

Sorry, I get carried away....Should be a nice, large DUMP.....Right on /. and the DPC...(hehe)

Hold on, ViRGE is raising some questions.......

Edit: Back up! WoooooooooooooMooooooooooooooooo!

 

[lordugh]

I was able to flush.. and then i wasnt.. and then I was... it looks like either he is beigh flooded, or there are still problems.

But above all iTS GREAT THAT THE PROXY WORKS AT ALL!

 

[CLL Sr]

I still can't connect. He will probably get it figured out when he gets to work. (I hope)

 

[BoberFett]

That machine must be getting hammered right about now. I hope we get the new box up and running soon. :Q Here's a clip from my pproxy logs, is this what others are seeing? It appears to be making the connection, sending a few finished blocks, then I get bumped.

07/03/00 21:56:16,Status: Uplink needed... (thresholds exceeded)
07/03/00 21:56:16,Status: Creating uplink (outgoing server) connection.
07/03/00 21:56:17,Uplink: [1] connection to 204.209.128.157 changed state
07/03/00 21:56:17, from 0 (ESTNONE) to 1 (ESTCONNWAIT)
07/03/00 21:56:18,Uplink: [1] connection to 204.209.128.157 changed state
07/03/00 21:56:18, from 1 (ESTCONNWAIT) to 50 (ESTREADY)
07/03/00 21:56:18,Client: 199.199.174.109 (client ver: 448) is connecting
07/03/00 21:56:18,Uplink: [1] Establishing handshake with 204.209.128.157...
199.199.174.109
07/03/00 21:56:34,Uplink: [1] Server is running version 313.
07/03/00 21:56:34,Uplink: [1] Got permission acknowledgement.
07/03/00 21:56:34,Uplink: [1] The perproxy says: "[pp] Team AnandTech pProxy!"

<snip>

07/03/00 21:57:28,Uplink: [1] 204.209.128.157 disconnected (reading).
07/03/00 21:57:28,Uplink: [1] Successfully put back 15 unsent blocks.
07/03/00 21:57:28,Uplink: [1] Closed connection to 204.209.128.157.
07/03/00 21:57:28,rc564 r=7985/20880, d=581/1000, 15.9 Mkeys/sec, tot=350
07/03/00 21:57:28,Status: rc564: Last uplink sent 52 and received 0

 

[Mika]

I think the pproxy is still acting up. It is basically consuming 1 entire cpu and using about 35MB of ram. The pproxy is still serving blocks, but seems quite slow.

Anyone from have any ideas what is happening? (d.net guys?)

Some more pproxy questions:
1. When you issue ctrl-break in the console to reload the ini file, does this act as if the pproxy had been restarted - ie do you have the 30min &quot;wait&quot; before you can buffer more than 1000 blocks? The reason I ask is that I'm chaning setting in the ini, and would like to test the settings without having to drop down to 1000 blocks.
2. Is it possible that the &quot;timeaverage&quot; and &quot;timeavgchunk&quot; settings are causing the high cpu usage? They are currently set to default, but what settings would potentially reduce cpu load?
3. Does dupe-checking consume significant cpu? I have temporarily disabled dupe-checking on the pproxy. So far, I haven't noticed any improvements on ram or cpu usage.

In short, I'm looking for any tips on improving the performance and decreasing the amount of resources the pproxy is using.

Mika

 

[Mika]

OK, I think I have a little better idea of how the pproxy works. If anyone knows otherwise, please let me know (hopefully some of the d.net guys are reading this).

I believe that at some point, the pproxy loads all the blocks into memory. It appears to happen immediately after it establishes a connection to a keyserver. This causes the huge increase in RAM usage. It is possible that while in memory, the pproxy does some processing (I originally thought it was doing dupe-checking, but turning it off made no difference). Hence the enourmous amount of CPU usage. In the state the connection to the keyserver is in, the pproxy doesn't seem to download any blocks, thus, all the clients are starved for blocks, and quickly use up the pool of open connections.

Here's what I did. I copied the buffout.rc5 file to a different machine (now 30MB!!) and started dumping from there (don't ask me why the other machine doesn't seem to have a problem flushing those blocks). Then I backed up the buffout.rc5 file on the pproxy (so it was not sitting in the original dir). After restarting the pproxy, it seems much more responsive, and seems able to serve clients better.

The bad news: I didn't start flushing the buffout.rc5 file to very close to the day rollover. Very few blocks will show up in tonight's d.net stats run .

There is a lot of speculation in the above, but it's all I can come up with right now. I would certainly appreciate some more knowledgeable folks to add more information to this.

Mika

also: there is about 500,000 blocks being flushed.

 

[Jator]

Mika,

I'm not help with the problem, but I just want to thank you for all your trouble in maintaining the pProxy. Mine is only a fraction of the size of yours, and I can see now how it is time consuming in toruble shooting it.

Jay

 

[Engineer]

Thank you God....I mean Mika....Really!!!

...May the D.Net gods favor you...

 

[Moose]

You can bypass the 1000 block limit by using

[rc564]
expertmode=1

same for all contests.

As far as the large buffer file. the proxy does not clean up after itself and the out buffer file is due to a large build up at one time in blocks. What you did was the right thing. Shut down the proxy. move the buff out file to another location and start the local proxy back up. configure another machine or configure another instance of the proxy and do not allow it to accept incoming connections and allow the proxy to flush completely.

i suspect the large buffout was the problem. so you know not alot of fullservers have larger buffout files than a few megs. I suspect you had a very large backup at one time.

moose

 

[Jator]

So what you are saying is....Mika was constipated!!!!

Jay

 

[ViRGE]

Well, we'll have to see if we can't find some good remote admin software then. If we can keep this from happening again, we won't have to suffer the downtime nor the upload pains.

 

[Engineer]

On top of that, Drakkhen dumped a load that stopped up the toilet...

I hope someone gets a remote &quot;plunger&quot; soon....

I can't hold back these massive &quot;loads&quot; anymore

 

[Jator]

My vote is for VNC. Regardless of what you choose, I will allow the stats admins access to my computers via this route once the admins have been chosen.

Jay

 

[dvch]

Another &quot;Thank You.&quot; to Mika for all of the time and effort he puts in on behalf of the team.

 

[imported_Thunder]

I seem to still be having a weird problem....

None of the machines that I have with a .8007 Win9X/NT client seem to be able to fetch and flush to Mika. Yet the proxy (313) from my office appears to be doing just fine (judging by the block count).

I have the clients (&amp; the proxy) still pointing to 'mika.dhs.org'...

Other than that... could anyone imagine what might possibly be going on?

-Brian

p.s. They were all working fine until this last proxy crash.

 

[CLL Sr]

I am having trouble this morning after a healthy dump. I couldn't get blocks again. We really need to get the other server up and running soon.
Mika can you help a sick cow out?

 

[Clueless]

Looking at my pproxy log, looks like Mika is out of packets again as of 14:14 UTC!