Transfering File over network delay

[cgott42]

I have 2 PCs in 2 remote locations, both on Verizon FIOS 150/150 Mbps speed. Both PCs are connected via Logmein VPN.
One PC acts as the server, and it is running Win 7, 4GB RAM, 3GHz Pentiun e5700 and TLink Gigabit Network
The client PC is running Windows 10, 4 GB Ram , 2nd gen i3 PC, and built in gigabit network
Both PCs are running fine, and when I run speedtest both get ~150 Mbps both ways.

Problem is that when I transfer a file from the server to the client, I get about 40 Mbps (i.e. 5 MBps, where the speed varies during a file transfer between 3-6 MBps)
The files that I transfer are usually ~10GB, however I've tested 2GB files and get the same result.

I've also tried turning off the Logmein and transfer via TeamViewer and the times were even slower (~3MBps)

Any ideas to help get something close to the 150 Mbps that I get via speedtest are appreciated

thx

 

[freeskier93]

What type of hard drives? 40 Mbps is about the realistic max read/write speed you're going to get out of spinning hard drives.

 

[cgott42]

yep both are spinning SATA drives , what drives would I need?

 

[cgott42]

though if my HDDs are the bottleneck - how is the speedtest able to get 150 Mbps (doesn't it transfer an actual file down/up) to test the speed?

 

[VirtualLarry]

Speedtests use multiple threads and multiple data streams to test with. Single file transfers, are unlikely to max out a pipe.

Google "bandwidth delay product".

 

[ylin0811]

teamviewer transfer is based on proxing in between two different remote clients. i am not sure how logmein transfer works, but i believe it is the same.

if you want the transfer speed to go the fastest, set up a filezilla or free ftp server and use that to transfer files directly in between both computers. your transfer speed will vary based on the latency, but it will be way faster than going through a proxing solution over the internet.

 

[Gryz]

Not so hard to test what is going on.

1) Have 2 files on the same HDD on the source machine. Transfer both files at the same time. Make sure they both are written to the same HDD on the target machine. If your throughput doubles, then you know it is a problem with the TCP-implementation of Win 7. (Which I doubt).

2) Have 2 files on 2 different HDDs on the source machine. Not just 2 different partitions, no, 2 really different HDDs. If you have only 1 HDD, you could make one RAM Disk, put the source-file there, and use the RAM Disk as second disk. Do a similar thing on the target machine. Make sure the 2 files are stored on two different HDDs. Then start 2 file copies. If the file transfers are still slow, you know the HDDs are not the limiting factors. If the file transfer is suddenly twice as fast, you know it was the HDDs.

3) Another thing you can do is: make RAM disks on both source and target machine. But the file on the RAM Disk on the source machine. Copy the file over the network to the RAM Disk on the target machine. That should give you an indication how much the HDDs are the limiting factor.

 

[Gryz]

Speedtests use multiple threads and multiple data streams to test with. Single file transfers, are unlikely to max out a pipe.

Google "bandwidth delay product".

Are you sure ?

I thought that this problem was solved since TCP got its window scale option.
https://tools.ietf.org/html/rfc7323#section-2

It should be enabled by default in Windows 7.
https://en.wikipedia.org/wiki/TCP_window_scale_option#Windows

150 Mbps with any sub-second RTT should be a piece of cake with TCP and the window scale option.
Of course, this is just theory, I haven't tested TCP myself in ages.

OP, what is the RTT between the two machines ? (The ping between them).

Edit:
It seems that Microsoft has messed it up again. They don't use window scaling as they should. They had to add some of their own algorithms, that slow everything down. Thanks Bill.
Here is a pretty good article, explaining the issue. And even explains what you can do to improve the situation.
https://www.duckware.com/blog/how-windows-is-killing-internet-download-speeds/index.html

 

[freeskier93]

yep both are spinning SATA drives , what drives would I need?

though if my HDDs are the bottleneck - how is the speedtest able to get 150 Mbps (doesn't it transfer an actual file down/up) to test the speed?

Shoot, I always forget hard drive speeds are MB/s. Most spinning drives top out at about 30-40 MB/s in real usage, which is as well over 150 Mbps.

 

[ylin0811]

Speedtest is based on two streams.

The first stream is to identify the server closest to you. After the server has been identified, the connection will close with a bidirectional fin,ack.

The second stream is when the actual speed test occurs. Connection max out is through the use of tcp window scaling that is enabled by default in most operating systems.

 

[joutlaw]

Can you try not going through the VPN?

 

[[DHT]Osiris]

Bear in mind as well, speedtests tend to be *very* 'best case scenario', to the point of ISPs often spoofing/fudging how the data stream works in order to make them look awesomer. If you can get a consistent download/upload speed from a reputable source, that is what your true speed is going to look like. As a rough example, I know my internet connection gives me a 4.7MB/s download speed, from Steam, Blizzard's launcher, and torrents. It never goes over that, rarely under that. If each system can download faster than your current transfers, you need to look at what would be restricting upload from source. It could be artificially limited by ISP, and/or they're lying to you about what your speeds really are.

 

[cgott42]

Not so hard to test what is going on.

1) Have 2 files on the same HDD on the source machine. Transfer both files at the same time. Make sure they both are written to the same HDD on the target machine. If your throughput doubles, then you know it is a problem with the TCP-implementation of Win 7. (Which I doubt).

2) Have 2 files on 2 different HDDs on the source machine. Not just 2 different partitions, no, 2 really different HDDs. If you have only 1 HDD, you could make one RAM Disk, put the source-file there, and use the RAM Disk as second disk. Do a similar thing on the target machine. Make sure the 2 files are stored on two different HDDs. Then start 2 file copies. If the file transfers are still slow, you know the HDDs are not the limiting factors. If the file transfer is suddenly twice as fast, you know it was the HDDs.

3) Another thing you can do is: make RAM disks on both source and target machine. But the file on the RAM Disk on the source machine. Copy the file over the network to the RAM Disk on the target machine. That should give you an indication how much the HDDs are the limiting factor.

Results:

• When I transferred from 1 HDD to 1 HDD - At first, I transferred both via copy/paste within windows explorer (with the 2 PC's connected via Logmein Himachi VPN), however as soon as I started the 2nd transfer the first transfer stopped. Transfer speed was ~6.5MBps
• When I transferred from 2 separate physical HDDs to 2 separate physical HDDs. - I couldn't transfer both via Logmein (as the server computer is set to a single HDD (anyone know how to change that?) , so I transferred one via Logmein himachi /windows explorer and the other HDD file via TeamViewer - the transfer speed remained the same on both
• Ping speeds (via Logmein Himachi) varied wildly between 2- 12 ms, most common speed was 4-5ms

How do I setup to not go through the VPN (if that's the problem).