Diagnosing Network Throughput Issues

[Armitage]

Ok, here's the problem.
I've got a cluster of DEC Alphas connected by 100baseT switched ethernet.
The network throughput is good between the cluster nodes (about 94 mbits/s half duplex, about 88-89 mbits/s each way full duplex).
But the throughput from the server to the nodes is terrible. About 70 mbits/s half-duplex, between 20 and 50 mbits/s each way full duplex.
The benchmark numbers are obtained with netperf, using the tcp_stream_script

All of the systems are running Suse 7.3 with a 2.4.17 kernel.
All of thesystems are using a Tulip based network card. There are two different tulip cards usedon the cluster, but the problem doesn't correlate with that.
The server has two of these NICs, one to talk to the cluster, and one for talking to the rest of the world.

At this point, we've tried rebuilding the kernel with module NIC drivers instead of building it into the kernel. We've tried loading the alternate tulip driver (DE45?), but it wouldn't load properly.

Next on the list of things to try:
Try another cable between the server and cluster switch.
Try another port on the switch.
Try another network card.
The network cables (all of them) may be to close to the power cables. We could rearrange them, but why wouldn't it be interfering with the node-to-node connections?

Any other ideas?

 

[Armitage]

FWIW, we found the problem.
Some meatball cable-tied the cat5 from the server to the cluster to about 4 feet of an extension cord.

It works great now.

 

[CBuxton]

Amazing...Just goes to show just how much of an impact cabling has on network performance. Can't image how sensitive 1000BaseT would be.

 

[Tallgeese]

<< Can't image how sensitive 1000BaseT would be. >>

Ummm...you do mean 100TX, right?
Never underestimate the importance of the Physical Layer (which we affectionately call El Numero Uno in our shop!)

Just like a stack of blocks, everything else depends upon the bottom block (L1),
Yank out that block, and everything comes tumbling down around your ears.

 

[miguel]

<<

<< Can't image how sensitive 1000BaseT would be. >>

Ummm...you do mean 100TX, right?
Never underestimate the importance of the Physical Layer (which we affectionately call El Numero Uno in our shop!)

Just like a stack of blocks, everything else depends upon the bottom block (L1),
Yank out that block, and everything comes tumbling down around your ears. >>



There is 1000BaseT now. Gigabit Ethernet with a UTP connector.

 

[spidey07]

please for the love of GOD stay away from 1000 Base-T. if you so much as breath on the cable you'll get performance problems.

my .02

 

[CBuxton]

Exactly....1000BaseT must be SO sensitive. Leave gigabit speeds to fiber.

 

[Armitage]

<< Exactly....1000BaseT must be SO sensitive. Leave gigabit speeds to fiber. >>



Are there any reviews/documentation/anecdotes regarding this? We may be upgrading this cluster in a year or so, and were considering gigabit over copper.
Also, I saw some cable at best buy the other day advertised as cat6. It claimed to be better then cat5e for gigabit ethernet? What's the difference.

BTW: We have myrinet on this cluster, but it's proven to be a real PITA, and isn't working at the moment.

 

[miguel]

<< Exactly....1000BaseT must be SO sensitive. Leave gigabit speeds to fiber. >>



Fiber is very sensitive as well, though. If you are in an environment where you move the cables around, I would think copper would be stronger/cheaper than fiber.

 

[Tallgeese]

<< There is 1000BaseT now. Gigabit Ethernet with a UTP connector. >>

REALLY? WOW! Learn something new every day...
I was simply trying to verify which one we were discussing since the first post mentions "100T"

I wanted to check, because if they DID mean 1000TX, I also would tell them to run screaming, or switch to more fiber in their diet...

 

[TurboQuattro]

<<

<< Exactly....1000BaseT must be SO sensitive. Leave gigabit speeds to fiber. >>



Are there any reviews/documentation/anecdotes regarding this? We may be upgrading this cluster in a year or so, and were considering gigabit over copper.
Also, I saw some cable at best buy the other day advertised as cat6. It claimed to be better then cat5e for gigabit ethernet? What's the difference.

BTW: We have myrinet on this cluster, but it's proven to be a real PITA, and isn't working at the moment. >>



You cna usually run 1000bt over standard cat5 without a problem, we do it here. If you do run into problems go with Cat5E or Cat6 cable.

In my experience 1000bt is only slightly more sensitive to interference than 100bt. But of course YMMV