Fast Ethernet Wire-speed?

[Hoober]

Dually - Full duplex just means that you can transmit and receive over the same line at the same time...

So if you have a 100Mbit connection and you get 80% throughput on it then you're able to send one way that much bandwidth. Duplex doesn't double the amount of data that you send through a computer, it just allows you to send each way at the same time. So if you're sending files and synching backups, then you're sending communications both ways over the same line. Duplex just allows the computers to send information both ways as opposed to just one way.

 

[cautery]

CTR - I will have to go research the "Store and Forward" vs. "Cut-through" issue.

The following is the hardware/software setup:

-Switch: Netgear RT314 with v3.24 firmware and Boot Base 2.00 (just flashed it yeaterday)

-NICs: All workstations have Netgear FA310TX NICs using the built-in Window 2000 drivers (these are the only ones that "officially" and explicitly reference support for Win2K. I have the last drivers Netgear made for these cards, but haven't tried them yet.

All cards are set as follows:
Burst Length: 16 DWORDS
Connection Type: Explicitly set to 100BaseTx Full_Duplex.
Network Address: "Not Present" (set on the TCP/IP protocol)
Transmit Threashold: 256 bytes (highest available setting and the default)

WS hardware:
1- Tyan S1832DL Tiger 100 dual PIII-700s at 784, 512MB memory.
2- ASUS P2B-S with Celly 300A at 504MHz, 256MB memory
3- ASUS P2B-B with Celly 300A at 558MHz, 256MB memory
4- Same as 3
5- Not testing as it is connected via another hub. Abit BX6r2 w/ PIII-700 at 903MHz 256MB memory.

Testing program does not read or write to the hard disks, so I would suspect that HDD specs are not relevent.

TCP/IP Settings:

"Obtain an IP Address Automatically" (Switch is set for DHCP Server service)

Cabling:

Each drop is wired with CAT-5E ports (tool-less)
Cat-5E cable from drops to Patch Panel in data closet
24-port Patch Panel (CAT-5E 568B punchdown type)
Custom patch cables from patch panel to switch (CAT-5E cabling and modular connectors)

Software:
4 workstations running Windows 2000 Pro SR1a and patched current according to the M$ site.
1 workstation runninf WinNT 4.0 SP6a and close to patched current.

Testing throughput with NetCPS, which runs in a command.com window on each machine. One as the sender (-s) switch, and one as the receiver (with the sending WS IP as the arguement)

All four Win2K workstations average within 1-2% of each other on the tests.
The WinNT WS is a little slower, but it is connected to the switch via another hub, so I would expect it to be a few more % slower.

Any other specs that you need, just ask....

MDay:

Thanks for all the help guys!

 

[spidey07]

Doing pretty good considering the celerons. again 80% is really pretty good.

 

[1KrazyFool]

I get 10.0MB/sec (80Mbps) with netcps. This was between:

400Mhz Celeron, 224MB RAM, WinME
800Mhz P3, 512MB RAM, Win2000

both w/ 3c905b's - latest drivers running through a cheap-o Addtron 5-port switch.

Another program I use is iperf. It allows you to play with TCP windows sizes, use UDP, etc. and is cross platform. I get 90.0Mbps between my Win2000 box and a linux box (2.4.2) w/ a 3com NIC (same switch). This is consistent with the 11.5MB/sec FTP transfers I get from the Win2000 box to the linux box using Win2000 CLI FTP (fast ftp).

I suspect that NetCPS can't push data out fast enough, or something. Who knows.

 

[esung]

CTR: Netgear uses the conventional store and forward method on it's switching ports. I don't think any of the Netgear's product is using the cut through artchitecture.

 

[cautery]

1KrazyFool - Thanks for the link to Iperf! I'm playing with it now. The compiled Win32 version I have doesn't support pthreads, but I've set up multiple server and client shortcuts to run multi-sessions.

1 each one-way session (port 5001): Average 81.9Mbit/sec
2 each one-way sessions 9ports 5001, 5002): Average 81.7Mbit/sec (actually, I think if I could start them both at exactly the same time, the average would be even closer.

I'm going to go run one and two sessions each way to test the FULL DUPLEX implementation just to ee that it is in fact working.

Hmmmm 81.9-82.1% no matter what window size and packet size I set.... Though the packet breakup does vary a bit. Wonder what else I can do? I surely wish Win2K let you look at and change MSS/MTU...

More later...

 

[cautery]

Well.... FULL DUPLEX is working just fine... approximately 81.7Mbit in each direction.

Enough for tonight. I gotta get my new dual box spec'd out...

 

[CTR]

Run the throughput test between two hosts networked with a crossover cable. Compare it to the same test run with the hosts linked via the switch. Any difference? Let's see if the linksys' store-and-forward is causing the slowdown.

 

[cautery]

Took me a few minutes to make the cable...

Ran it peer-to-peer, and the results were about 81.9-82.1Mbit. It seems that the switch has next to zero latency.

Does Windows just have that bad of a TCP/IP stack?

 

[Shadow07]

Cautry, you are doing fine with 81Mb/s each way. And no, Windows does not have a bad TCP/IP stack. You need to keep in mind that Windows has a very resource intensive GUI that will slow down transmissions.

 

[spidey07]

au contrare (sp?)

Windows IP stack is terrible. It does not follow the RFCs for the TCP algorithym. It does not use a sliding window no adjust the window size accrodigingly (this leads to VERY ineffecient data transfer). It up and queries every name server configured all at once. Keep in mind that this is MICROSOFT TCP/IP. It is not a true tcp/ip stack. Just microsoft's interpretation of what a stack should do without regard to all the RFCs governing its behavior.

That's really my biggest gripe with the stack. The tcp window just stays stuck at 8096 and never increases.

<edit> forgot to mention that 2000 takes care of most of these problems. Also uses large TCP windows of upto something like 2 megabytes. that helps.

 

[Shadow07]

I would agree with you on the earlier versions, but with 2000, it is much improved. What I was saying is that with his transmission rates, he is going to be getting the best for Windows 2000. Most of the overhead is caused by the MS GUI, not because it is a BAD stack. It does not compare to UNIX/LINUX stacks when it comes to security, but with reliablity, it is good. But, if you do want to change the window, you can edit the registry to change this value.

 

[cautery]

Don't think I want to hack on the registry or anything. 81Mb/s will likely handle anything I need to do for the foreseeable future... I just don't like inefficiency in anything.

I'm planning on bringing my web, email, ftp, AND DNS in house if 1)PacBell will let me mod my DSL order to Enhanced w/ 5 IP addresses, and 2)they don't try to charge me for providing DNS while I get all my hardware/software together. I don't have a whole lot of traffic these days, so it'd be a lot cheaper to host everything from home, even with the higher priced DSL.

I'm not understanding how the GUI can suck up so many resources that it eats 20% of the TCP/IP bandwidth..... I'd like to know how that happens....

Do they make NICs that provide onboard processing that could offset the Win2K deficit... kinda like the new Intel NIC does with encryption.... unloading the processor from encryption duties.... Just throwing it out there....

20% because it has pretty pictures.... Dang that's a pretty hig price to pay....

Later,

 

[Shadow07]

Well, you almost have it. Yes, because it has pretty picutres. Also, try using the IE FTP option. When you try to copy over files off of an FTP server, IE will calculate all of the files and the total file size. The algorithm it uses takes FOREVER if you are trying to copy over lets say 100MB+. Then try using FTP Voyager to copy the same amount of data. You will see a better performance from FTP Voyager. But, if you are going to host a web server, you will be fine. Your users should not experience any difficulties with IIS5.

 

[1KrazyFool]

Have you checked your CPU % using task mananger on any of those boxes? I see about 20-30% when doing a full duplex test (two sessions).

 

[Shadow07]

With how much memory available, and total amount of physical RAM?

 

[cautery]

Regardless of the direction or number of threads:

Single CPU (128MB memory) 63-68% utilization
Dual CPU (512MB memory) 26-29% utilization

 

[Ender78]

The reason that you are seeing only about 80% of the 100Mbps is due to the properties of Ethernet. I wish I could remember the exact reason, but Ethernet just starts to saturate and now work past 70-80%. Token ring (although slower, is able to work at upto 90% of its stated capacity).

 

[cautery]

Ender78 - You may well be right about the 80% thing in Windows, but I KNOW for a fact that they got near wire-speed in some of the original Beowulf clusters using FA310TX cards (or other tulip chip based cards), tweaked drivers (the ones currently used in most Linux distros) and Linux.

I just want to make sure that I can't easily squeeze more efficiency out of the Win OS before I give up. At some point I will be moving all my web, email, ftp, and DNS stuff onsite, and at that point, I will likely move those functions over to a *nix platform of one flavor or another.

Later,

 

[Ender78]

These numbers come from pre or very early windows days (yeah we had networks before we had windows). The speed is not as much the processors speed (but now that I think about it, it may be). The computer just cannot deal with interupts at wire speed (with a Multi proc server I guess you can get that extra 10-15%, the rest will just be lost in the medium).

 

[cautery]

Ender78 - You see, that's where I get confused. If it is a processor limitation, then why did the Beowulf guys achieve near line-speed on their systems? They were using single processor systems with slower processors than I hahave now.... a lot slower in fact. Some how, I can't see it being an interrupt volume problem.

I'm also inclined not to think it is a processor utilization issue either... since at 80% of wire-speed on my single proc system, I was only seeing max 68% proc usage (though that seema awfully high). What I'd like to see is a NIC with an onboard processor that would unload the CPU and handle all the I/O from disk and memory. Though this may not be possible depending on the bus architecture.... I don't know. That's not my area.

Later...

 

[Shadow07]

What version of Windows are you running again? If it's 9x then I can see a high CPU utilization, because that version of the Windows Kernel does not utilize the CPU correctly. It handles more of the frame/packet creation and ECC than that of the NT code.

 

[cautery]

It's back on page one of this thread where I list all the hardware/software: Windows 2000 Pro w/ SR1a and patched current.

Thanks,
Clay

 

[ScottMac]

The speed of the PCs, the Manufacturer and model of the Network Interface Card, the quality of the cable &amp; connectors will all affect the overall throughput.

Even if you have a 1.2 Gig Athlon, if you put a cheap NIC in it, you'll never see ultimate throughput. That's why cheap NICs are Cheap.....they suck perofrmance-wise (though they may be &quot;great performance for the price&quot.

Lots of folks go out &amp; buy pretty decent CAT5,5e, or 6 cable...then crimp on UNRATED mod plugs...effectively making the cable CAT 3 (weak link rules apply..the path is only rated as high as it's lowest rated component). Some folks don't know that there are different mod crimps for solid-conductor and stranded-conductor cable...using one type on the other type cable may reduce the category rating (if you get a good connection at all).

Also note that a &quot;Wire Speed&quot; switch may accomplish &quot;Wire Speed&quot; through buffering...increasing latency and causing problems for real-time applications. Most vendors have their own definition of &quot;Wire Speed&quot;...read carefully.

Overall network speed is also going to be affected by the protocol in use. If the application is using TCP, the sending machine must eventually wait for an ACK packet from the receiving machine before it can continue to transmit (making it look &quot;Slower&quot. UDP packets are sent with no expectation (at that layer) of any acknowledgement...so they tend show as &quot;faster&quot; (and used for testing/benchmarks, since it presents better throughput figures).

Most switch tests are done with products like a &quot;Smartbits.&quot; This is testing apparatus (and software) specifically designed for throughput, latency, and packet drop testing (and many other functions). Because of the way the Smartbits sends and receives the data, it will always show better throughput figures than a couple PCs blabbing at each other though a switch.

If you want to try a pretty slick visual throughput tool, find a copy of &quot;Bricks.&quot; It was written by DEC (Digital Equipment Corp, now defunct) guys for use at trade shows to show throughput visually for demonstrations. It can be set for TCP or UDP packet types. basically it has a pile of &quot;Bricks&quot;...each &quot;brick&quot; contains X number of bits (1Meg, 10Meg, etc)...as each increment of bits get transmitted, a brick is pulled off of the transmit pile and added to the receive pile. Multiple transmitters can send to a single receiver to show reletive aggregation ratios (stream / stream). The program is Public Domain and shareable.

FWIW

Scott