HORRIBLE performance? - Gigabit Ethernet

[mamisano]

Hey all,
Just installed a Trendware Gigabit NIC (Realtek Chipset based) into my home server, and a TrendWare Gigabit 8-port switch. My Abit motherboard has onboard 3Com 10/100/1000. Performance between the two is HORRIBLE.

Using Sandra 2004:
Latency: 113us
Speed: 9392KB/s (76939kbit/s)

This is essentiall 100 speed, not 1000.

I know my choice of hardware is not top-notch, but I thought I *might* get something out of the upgrade....If not, looks like I will have to pick up a 64-Bit card off of Ebay for the Server.

 

[AFB]

What OS ?

 

[mamisano]

Windows 2003 Server
Windows XP SP1

 

[volrath]

Does windows list each of the connections as 1000mbps?

 

[WannaFly]

Make sure you have a cat5e or cat6 cable. This is typical gigabit over copper speeds though(IIRC). Force the speeds on each card - you might get a little better results

 

[mamisano]

Both PC's have Cat5e and say 1Gbps...

 

[n0cmonkey]

What does sandra use to measure throughput? Have you tried just FTP? Maybe switch the cables, and see if that helps.

 

[mamisano]

Via FTP: 21479.59 KBps

Will try new cables in the morning. I have a TON (80) of new Belkin Cat 5e cables in the basement.

Edit:
What is up with SANDRA 2004? When I test the connection, and view Task Manager, the meter reads 0.05% Utilization...???

Transferring via FTP, meter reads around 16-17% on both computers and 10-11% CPU Utilization.

Server has 2x WD 250GB in Raid 1, TX2000 Raid card in 66Mhz slot.
WS has 2x WD Raptor 36GB in Raid 0

I can transfer 2 LARGE (ISO Images) files at the same time and that about peaks out my transfers. Meter goes to 25% on both. Any more files and Dual WD Raptors start thrashing, throughput drops to 18%

 

[cmetz]

mamisano, the Realtek chipset sucks. It sucked in the 10/100 RTL8139 design, and guess what? The MAC core and bus interface for the RTL8169 are exactly the same design that sucked for 10/100. I have the same Trendware board you do. It's junk. Use it like a mediocre 10/100 board.

The 3Com 940 chip is really the Marvell Yukon chip. I had high hopes for this one (in turn, it's based on the SysKonnekt core which has great performance) but it's somehow not delivering in my tests, it's ending up with fair performance but not good. It's okay but not going to deliver impressive performance.

The Netgear GA302T is the best board I've tested thus far for 32 bit 33MHz PCI performance. They run about $35 shipped on the deal of the week (forget where).

64 bit may help or may hurt. Believe it or not, some boards perform slower on 66/100/133MHz or 64 bit than they do on normal 32 bit 33MHz PCI - probably because they're optimized for the latter case. The 3c996BT is the same Broadcom chip as the Netgear GA302T but with a PCI-X bus interface and should be a good choice for 64 bit - a lot of server vendors put the Broadcom gigabit chip on board.

What CPU and chipset are you using? Some of them deliver really poor gigE performance, period. In my testing, all Athlon platforms lag the P4 platform, for example (I don't yet know why, though I can speculate).

 

[Amused]

Originally posted by: WannaFly
Make sure you have a cat5e or cat6 cable. This is typical gigabit over copper speeds though(IIRC). Force the speeds on each card - you might get a little better results

My Gigabit connection shows 40 MB/s on Sandra between all my computers. Same at my brother's house.

mamisano's results are NOT "typical."

Gigabit is getting a bad rep because people with slow HDDs cannot understand why their files wont transfer much faster. They don't seen to realize that HDD speeds become the bottleneck with gigabit ethernet. Use Gigabit on a netrwork of systems with RAID-0 Rapter or SCSI arrays and watch file transfers fly. Use it on a system with a mix of old, slow drives and watch it crawl.

 

[mamisano]

Originally posted by: Amused

Originally posted by: WannaFly
Make sure you have a cat5e or cat6 cable. This is typical gigabit over copper speeds though(IIRC). Force the speeds on each card - you might get a little better results

My Gigabit connection shows 40 MB/s on Sandra between all my computers. Same at my brother's house.

mamisano's results are NOT "typical."

GB is getting a bad rep because people with slow HDDs cannot understand why their files wont transfer much faster. They don't seen to realize that HDD speeds become the bottleneck with gigabit ethernet. Use Gigabit on a netrwork of systems with RAID-0 Rapter or SCSI arrays and watch file transfers fly. Use it on a system with a mix of old, slow drives and watch it crawl.

Server has 2x WD 250GB in Raid 1, TX2000 Raid card in 66Mhz slot.
WS has 2x WD Raptor 36GB in Raid 0

 

[Amused]

Originally posted by: mamisano

Originally posted by: Amused

Originally posted by: WannaFly
Make sure you have a cat5e or cat6 cable. This is typical gigabit over copper speeds though(IIRC). Force the speeds on each card - you might get a little better results

My Gigabit connection shows 40 MB/s on Sandra between all my computers. Same at my brother's house.

mamisano's results are NOT "typical."

GB is getting a bad rep because people with slow HDDs cannot understand why their files wont transfer much faster. They don't seen to realize that HDD speeds become the bottleneck with gigabit ethernet. Use Gigabit on a netrwork of systems with RAID-0 Rapter or SCSI arrays and watch file transfers fly. Use it on a system with a mix of old, slow drives and watch it crawl.

Server has 2x WD 250GB in Raid 1, TX2000 Raid card in 66Mhz slot.
WS has 2x WD Raptor 36GB in Raid 0

I wasn't saying your results were because of slow HDDs. HDD speeds have little effect on the Sandra test results because the file size Sandra uses is so small. Your results are because something is wrong with your network.

I was making a point about the beating Gigabit has gotten on tech sites from users who can't transfer their 40 GBs of pron from one system to another much faster than 10/100 because they are running slow HDDs. They then start spreading the BS that gigabit is not much faster than 10/100.

If you have the HDDs to support the fast file transfers, gigabit can be 4-5 times faster than 10/100.

So don't let people tell you those results are typical. They are far from it.

 

[mamisano]

Thanks. I have a second Trendware card here and I will pop that into the Server, see if that changes things. If not, I will put the Trendware card into the Workstation and again try the tests.

 

[ScottMac]

I'd say try some commercial cables (but I always say that).

They way many people fabricate cables, they may as well just use coat hangers.

If the cables are not up to snuff, you'll never get any kind of performance.

FWIW

Scott

 

[Amused]

Originally posted by: mamisano
Thanks. I have a second Trendware card here and I will pop that into the Server, see if that changes things. If not, I will put the Trendware card into the Workstation and again try the tests.

BTW, just so you know, my network uses a mix of Intel pro cards and D-link DGE-530T gigabit cards and a D-Link DGS-1005D 5 port gigabit switch using generic Cat5e cables from Newegg. All systems show a consistant 40MB/s on Sandra. Real world transfers are directly effected by the slowest HDD array between the two systems in the file transfer.

This may very well be a problem with your trendware cards. Can you go pick up a couple Intel cards and try them?

 

[JackMDS]

If your Switch supports Jumbo frames change the MTU of the two computers to 9000.

Under "normal situation" between two home computers on a peer to peer Network. The gain of "Speed" might be only 25% to 50% (x .25 to .50).

I.e. if your system is 100Mb/sec. and yields 10MB/sec. transfer, by replacing the NIC with Giga NICs and the Switch with Giga Switch the Network will probably yield 12.5MB/sec. to 15MB/sec. (if your hardware is fast).

If one of the computers is equipped with Real Server OS like Windows 2003 the "Speed" from Server to client might improve by 150% i.e. you will get 25MB/sec. (x2.5)

If you install Giga on Double Xenon Computers with fast SCSI RAID, and Server Software you might get 400% (x4) improvement.

All numbers are approximation for demonstration purposes, YMMV.

Link to: The Big Giga Thread

:Q

 

[Amused]

You know, Jack, that just isn't my experience. In real world transfers of large files, I see 2-4 times the speed with gigabit over my old 10/100 network speeds.

My hardware is listed in my rigs link. Of course, the transfers to or from my old Dell are much slower. But transfers between my 2.0a and 3.2c machines with RAID-0 arrays are much faster with gigabit. What used to take an hour to transfer now takes around 15 minutes.

 

[JackMDS]

My personal experience is a long the lines that is in my post.

I am following a lot of Giga Stories concerning "Normal Systems" (one CPU with a variant of Windows), and can not make a sense out of it.

I use the term "Normal" since there is one report from Sun talking about 800Mb/sec. out of 8 CPUs Solaris.

 

[cmetz]

JackMDS, sometimes jumbo frames speed things up, sometimes they slow things down. Depends on enough factors that it's best to call it voodoo. I usually recommend folks try performance at 1500, 4096, 8192, 9000, 9180, and 9200, and see what works best in their particular situation.

Variables include hardware bugs (many network interface chips have bugs in their jumbo frame implementations such that performance-enhancing features must be disabled if jumbo frames are enabled), OS memory management, page sizes, cache line sizes, and PCI burst lengths.

This is partly where the "we don't need jumbo frames because they don't speed things up" arguments come from - owing to small scale usage, there's a lot of jumbo frame problems that rob performance, and a lot of the gain that should be there isn't always realized.

 

[chsh1ca]

You have other problems than your network. You should be averaging over 10MB/s on 100mbps, peaking at 12.5 -- that's what I get here with a linux file server hooked to a linux desktop through a 10/100mbps switch. A crossovered Gbps NIC in both machines gives me about the peak for my drives (~95-100MB/s). The difference may be that I have name brand equipment all around, 3com NICs everywhere, decent ASUS motherboards in the machines in question. The key weak link may be the Ovislink switch. This is transferring via ProFTPd from the file server using wget on my desktop client and timing the tme ti takes to establish the connection and transfer the file, which only falls on the conservative side of things, the transfer will be a a second or two faster.

It would be good to see the OP's configuration of both machines (drive speeds, how the drives are connected, etc). And have you actually tried a transfer, and timed it, or just a benchmark?

 

[JackMDS]

Originally posted by: cmetz
JackMDS, sometimes jumbo frames speed things up, sometimes they slow things down. Depends on enough factors that it's best to call it voodoo. I usually recommend folks try performance at 1500, 4096, 8192, 9000, 9180, and 9200, and see what works best in their particular situation.

I am sitting now by the computer seeing the Top of the Empire State Building through the Windows. For some reason a thought flashed through my mind. Jumping from the Empire State Building is dangerous. Playing around with the MTU on an Humble Home Network is not. I Can always change back.:Q:brokenheart:

 

[mamisano]

Hmmm,
I plugged the Ethernet cable into the onboard Intel 10/100 NIC on the Server. When I run Sandra, I get 9 or 10MBps and it comes up Instantly.

I moved the Trendware card to another PCI slot, re-ran the tests and Sandra hangs for about 1 minute and gives me 146kB/s

Going to pull the card and replace it. If that doesn't work, there is an Intel Pro 1000/XT card on Ebay that I am eyeing!

 

[Amused]

Originally posted by: mamisano
Hmmm,
I plugged the Ethernet cable into the onboard Intel 10/100 NIC on the Server. When I run Sandra, I get 9 or 10MBps and it comes up Instantly.

I moved the Trendware card to another PCI slot, re-ran the tests and Sandra hangs for about 1 minute and gives me 146kB/s

Going to pull the card and replace it. If that doesn't work, there is an Intel Pro 1000/XT card on Ebay that I am eyeing!

Well, it's good to hear you found the problem.

 

[mamisano]

Same problem with the 2nd card. I guess this board does not like the Realtek chipset...

Oh well, just picked up an Intel Pro 1000/XT 64-bit card from Ebay... $40 shipped. I will post results when it arrives. In the meantime, I will try the NIC in a workstation.

 

[gunrunnerjohn]

FWIW, I don't get much improvement with two 2.4g P4 machines using Intel Pro/1000 MT boards over the old Netgear 100mbit boards when writing to the remote machine, about 30%. I get about 100-150% better speeds reading from the remote machine, but the real requirement is writing. OTOH, when I write to a Server 2003 machine, I get close to 30mb/sec write speeds. I believe most of the speed issues are SMB bottlenecks on Windows workstation machines based on my experiences. I get 500-550mbit/sec raw network speeds with QCHECK both ways, so I don't think it's the transport.