Data Transfer is Slow as Molasses

[Carbo]

Trying to transfer a 30GB file on a home network from a desktop to a laptop.
The file is a mix of Word docs, some small video files, and many photos in JPEG format. The transfer rate is around 6MB/sec.
The desktop is running XP, the laptop Windows 7 Pro. Cable connection via Comcast. Router is a Netgear WNDR 3700.
Connection when surfing the internet is strong and fast.
Any ideas why the transfer rate is so slow? Any suggestions?
Thank you.

 

[JackMDS]

If you are transferring through Wireless that is the way Wireless does it.

If you would use Giga Wire connection you might get it x10 faster.

 

[Carbo]

Jack, I'm new at this, but are you saying the transfer rates I'm getting are the norm?
Why is there no difference when I ran the laptop on a wired connection?

 

[bobdole369]

6MB/sec is roughly 60mbits. Pretty much the top speed on a 100mbit network. Its how fast 100mbits goes. You might get up to 80mbits with a little cajoling but essentially thats the speed it works at. Gigabit ethernet is 10x faster, so the limit there is the speed of the hard drives, typically you can transfer 25-40MB/sec without optimizing anything.

 

[cheez]

Carbo, that still sounds a bit too slow for hardwire 100mbps connection.

Culprit:
- Your WinXP (it's notorious for slow data transfer connection in the network), so this can be the bottleneck. Probability 10%
- Check and make sure you are REALLY getting 100mbps, it might be running at 10mbps. Go to "Show all connections" and go into the Local Area Connection. Probability 30%
- You are transferring large number of files, as many as ten thousand or more rather than transferring a few large files. There is a huge speed difference between transferring two 15GB files vs. 15000 files that is worth 30GB. Probability 90%, lol.


.

 

[Carbo]

cheez, checked some of your suggestions.
According to my Local Area Connection the desktop running XP is at 100Mbps.
The folder I was attempting to transfer is approx 30 GB's, 1,400 folders and 13,000 files. A combination of MS Word, PDF's, approx 6GB's of high resolution photos in JPEG format, and about 10 GB's of video files.
That said, I'm still surprised the transfer is so slow. I'm looking at about four hours to complete the task. Even more surprising is that plugging the laptop directly into the router made no difference.

 

[Fardringle]

Windows does a lot of "chatter" between individual files when copying or moving files on a network, so when you copy a folder full of many small files your transfer speed will be significantly slower than if you were transferring large files.

If you don't want to just wait for all of the little files to finish copying at the speed you are getting, you can zip all of the folders and files into a single (or a few) large archive file and then transfer that file to the other computer. Depending on the speed of your computers, the time that it takes to compress all of those files might not make up for the difference in transfer speed.

Another option is to copy the files to an external hard drive or flash drive and then attach the drive to the laptop and move the files onto the laptop hard drive.

 

[Carbo]

Yeah, I think the best approach is to copy the files onto a flash drive and then copy them over to the laptop.
Thanks for all the help, folks.

 

[JackMDS]

This post is just a description of the state of the issue at hand and is Not directed toward any individual in specific.

Businesses for many years are using 100Mb/sec. Networks, and most of them still do.
They transfer more files than End-Users and do not spend their time on none stop complaining about it.

We need to do what we to do according to the available technology.

Flash Drives are slow too. To best solution is to keep handy a 2.5" hard drive in a enclosure and USB or SATA and use it in such occasion.

Thus, it comes down to three choices.

1. Wait for the network

2. Use an external drive

3. Keep complaining.

 

[Nuwave]

Yeah, I think the best approach is to copy the files onto a flash drive and then copy them over to the laptop.
Thanks for all the help, folks.

Considering the 13000 files, I bet you $5 that this takes longer than letting the network do the work.

 

[Mr. Pedantic]

Considering the 13000 files, I bet you $5 that this takes longer than letting the network do the work.

This. Plus, doing it over network means you can go do something else. Whereas you have to manually oversee the entire transaction if you do it with a flash drive.

 

[Modelworks]

I still haven't upgraded my entire home network to gigabit , but I can tell you that transferring a file from my windows 7 pc to my windows server I get 11.59MB/sec for a 100Mbit connection.

The other thing is don't use explorer to copy large amounts of files over a network, it has a lot of disadvantages. Instead use something like teracopy, it is free.
http://www.codesector.com/teracopy.php

 

[cheez]

Windows does a lot of "chatter" between individual files when copying or moving files on a network, so when you copy a folder full of many small files your transfer speed will be significantly slower than if you were transferring large files.

If you don't want to just wait for all of the little files to finish copying at the speed you are getting, you can zip all of the folders and files into a single (or a few) large archive file and then transfer that file to the other computer. Depending on the speed of your computers, the time that it takes to compress all of those files might not make up for the difference in transfer speed.

Another option is to copy the files to an external hard drive or flash drive and then attach the drive to the laptop and move the files onto the laptop hard drive.

The "chatter" part with the individual files is absolutely correct.:thumbsup: However zipping or rarring them won't help...as the computer still has to check through every single files inside it. The process time for the transmission would be just as long.. I used to work for a company that does data management and backup's. We have files that are as many as over 100,000, all zipped, but takes hours and hours to get them transferred in the wired network. Flash drive might not be good either, due to the limited speed of the memory access in the drive.


Transferring huge number of files not only congests the network traffic but also the speed is affected by what OS you are running.. as well as the CPU and memory. Transferring files from hard drive to hard drive would be best, but also opting with Windows Server 2003 OSes or newer on the PC's will boost this performance effectively. Using gigabit lan would certainly help a bit too if using lan line is the only option.


.

 

[Fardringle]

The only reason a large zipped archive containing many small files would take longer to transfer is if one (or both) of the computers scans each individual file inside the archive for viruses during the transfer process. However, on fast computers, this wouldn't add much time since relatively decent computers can scan files much faster than a 100mbps network can move them.

As a simple test so I can give exact numbers, I just zipped up a 1.2 GB folder containing a bit over 12,000 files (my Adobe Lightroom catalog files). On my crappy old computer (single core Sempron running at 2.0 Ghz) it took 14 minutes to zip all of the files into a single archive. The resulting archive was 1.18 GB since the catalog files don't compress, which is why I chose this folder so the total data transferred is the same for the tests. My 100mbps home LAN transferred the zipped archive from one computer to another in 3 minutes and 48 seconds. Total of 18 minutes to move all of the files from one computer to another.

The same LAN connection and the same computers took nearly 40 minutes to transfer the entire folder of individual uncompressed files.

This is an extreme example since I used a very large number of very small files, but it shows how much of a difference the overhead chatter in Windows can make when copying multiple small files. It's not always efficient to take the time to compress the files before transferring, but in some cases it can make a significant difference, which is why I mentioned it as a possibility, while at the same time I said that it might not help depending on the situation...

 

[JackMDS]

@Fardringle

Nice post. :thumbsup:

 

[Gooberlx2]

Richcopy is also a nice tool for something like this.