Gigabit ethernet too SLOW

[eng530]

I am attempting to transmit a 42 MB/s digital video stream from a Firewire camera over a dedicated point to point gigabit Ethernet connection.

We are currently just testing the point to point gigabit Ethernet connection and are having problems generating enough throughput.

Si-soft Sandra benchmarks indicate that 50 MB/s is achievable with our connection (cat 6 cable, jumbo packets, flow control on). And the HD's are capablt of 93 MB/s

However we have been physically testing the link by transferring a large (1.5 GB) file and can only achieve approximately 20 MB/s (25MB/s - simplex and 20MB/s + 12MB/s duplex).

Why is this throughput limited?????

At present we have a 2800XP (shuttle) connected to a 2.4GHz Celeron, both with D-link DGE 530T Gigabit Ethernet cards, 1 GB ram and SATA raid 0 Hard drives (2xBarracuda 80GB NCQ and 2xMatrox Diamonmax9 80 GB).

Cheers

 

[sharkeeper]

What's the CPU utilisation look like?

Bandwidth sharing? Chipset INF updates?

We can sustain about 115 MB/S for hours but the setup is optimised for this. (iSCSI)

 

[eng530]

The CPU utilisation ranges from 30 to 70% usually. The system recieving the data seems to have the highest utilisation (I would have thought this would have been reversed?)

There is no sharing. Two dedicated computers with a dedicated cable.
Drivers should be up to date.

What type of system are you using to achieve 115MB/s.
what spec is your PCI bus to achieve this???

 

[Acanthus]

It may be a limitation of the camera, the HD could also be a bottleneck, dont trust sisoft sandra for HD benches, It tells you the PEAK transfer rate, not the average.

 

[kextyn]

I usually get low speeds over gigabit when transferring a lot of files or large files because the computer I transfer from is only an Athlon 1200 with 512MB of RAM. But I typically get around 150-210 mbps. With those systems I don't see why it would be slow. What networking hardware are you using?

 

[Goi]

It could be several reasons. Like Acanthus mentioned, your camera and HDD might not be fast enough. Also, if you have a PCI Gbe NIC, remember that PCI is a shared bus that has a peak of 133MB/s, which Gbe is exactly rated for, so you will never get the full bandwidth unless there's a dedicated channel for your Gbe controller of you're using PCI-X NICs.

 

[ribbon13]

Gigabit would be 125MB/s wouldn't it?

 

[sharkeeper]

What type of system are you using to achieve 115MB/s.
what spec is your PCI bus to achieve this???

SuperMicro backplanes, NIC and HBA on dedicated PCI-X at 133 MHz. Cisco switches.

 

[ribbon13]

what HBA and NIC?

 

[sharkeeper]

Intel Pro 1000MTx2 and SRCU42X HBA w/512MB cache in WB mode. Hosts use 1000MF with dual 320-4X's with 256MB cache each.

 

[VirtualLarry]

Crazy thought - instead of point-to-point GigE, what about point-to-point Firewire networking, using some Firewire 800 cards?

 

[PurdueRy]

could the firewire be the bottleneck? What is the limitation on those things

 

[Tab]

Originally posted by: ribbon13
Gigabit would be 125MB/s wouldn't it?

Yes, in theory that would be the maxium bandwidth.

 

[Appledrop]

try changing duplex mode in both NIC config... i had piss poor network performance over my marvell yukon gigabit ethernet before i turned it to half duplex mode

 

[eng530]

PC utilised are:

PC 1 Intel Celeron 2.4GHz CPU,
Abit BE7-S Motherboard with 512MB Crucial DDR3200 Ram,
2 x 80GB SATA Maxtor DiamondMax 9 in RAID 0 config,
Geforce 2 MX200 Graphics card

PC 2 is a Shuttle SK43G VIA KM400,
AMD Athlon 2800+ CPU,
512MB Crucial DDR3200 Ram,
2 x 80GB SATA Seagate Barracuda HD in RAID 0 config,
The graphics is built in.


Weve been trying various improvements.

removing virtual memory had no effect.

upgrading to 1 GB ram had no effect.

Moving the O/S from the sata raid to a third drive ATA66 drive (5400rpm 20GB) and reformating the sata raid0 gave he following improvements. (with 64KB clusters)

Shuttle to PC transfer rate increased from 20 MB/s to 37-40MB/s.

However PC to Shuttle transfer only increased from 20 to 26-30 MB/s. ????

(we have been testing throughout with a 1.5GB folder containing multiple mp3 tracks and also the same folder zipped. Initially the unzipped folder was always 10% faster however with the new configuration the zipped file transfers 10% faster).

 

[halfadder]

Firewire 800 is still only 100 MB/sec. In theory GigE would still be faster.

 

[sharkeeper]

Not to mention there is a serious limitation (distance) with fireWIRE.

We frequently send data from VA to CA at high speeds using GWANS. Not going to do that with anything else.

I suppose 1394 would work OK if the computers are in the same room though.

 

[bob4432]

what type of video stream is 42MB/s?

 

[halfadder]

Originally posted by: bob4432
what type of video stream is 42MB/s?

Good question. DV and HDV are only 3.125 MB/sec. And even DVCproHD is only 6.25 MB/sec.

 

[eng530]

Originally posted by: bob4432
what type of video stream is 42MB/s?

We are using a 6.6M pixel (3002x2210), 10/8 bit CMOS sensor which can output 5 frames per second in a pixelink firewire 400 package.

http://www.pixelink.com/products_info.asp?id=39

Originally posted by: sharkeeper
Not to mention there is a serious limitation (distance) with fireWIRE.

I suppose 1394 would work OK if the computers are in the same room though.

We did try an initial test with a point to point firewire 400 (PCI) data transfer but only achieved 4 MB/s , havn't got round to working out what the problems there were though.

Also we are looking at 3 km data transmission, so we need optical data transfer and I think its only an option on Ethernet.

 

[Bar81]

I know that many of the older firewire chips (and even now???) would peak *much* lower than the theoretical bandwidth; in fact, exactly at the speeds you were describing initially - low 20s

 

[Arcanedeath]

One of your systems has a Via chipset they are typicly limited to around 70-80MB/Sec on the PCI bus due to a poor PCI implemtation. Its possibly something is stealing the bandwith from that system and messing up your GbE Nic's, also I'd suggest changing the PCI latency timer to at least 64 and make sure that the GbE Nics are in busmastering slots that are not shared and have their own IRQ, also depending on what chipset your GbE Nic's use you may only get 20-30MB/Sec max transfer rate unless it's one of the better GbE chipsets, Personaly for GbE I wouldn't trust anyone besides Broadcom or Intel if its on the PCI bus.

Edit: could also be your cat 6 X-over cable, try a new Cat 6E cable and see if that helps too. Also as mentioned above unless you have a newer firewire chipset that connects to the camera you could be bottlenecked right there and if the firewire is also on the PCI bus your not gonna make it bandwith wise as the PCI bus at max is around 125MB/Sec in theory, in the real world however even the best 32bit 33mhz PCI tends to top out around 110 MB/Sec on a good chipset.

 

[ribbon13]

With the Sonnet FW800 on a 66Mhz/64-bit PCI-X I've gotten a 82MB/s peak 67Mb/s sustained transfer rate. But that serves you no help because the iLink on your camera is likely your problem. Perhaps it can't actually transmit that fast? Period? Have you tried contacting the camera manufacturer about it?

 

[eng530]

I've not actually got the camera at the moment. I have just been testing the gigabit ethernet link by dragging and dropping a large file into the shared folder of the other computer.

I was also concerned about the camera output being too much for the firewire 400, I'll contact the manufacturer and see what the say.