WD Black file copy from 4TB to 5TB only 64-66MB/Sec

[scgt1]

So I'm transferring my Steam drive from a WD Black 4TB to 5TB due to being down to 48GB. I'm doing a straight copy from one File Manger window and drop to another. (IE Windows copy) The specs for the drives are below. Far cry from what they should be doing a sustained transfer at. They are both connected to the 6GB ports on my MVIF with 6GB sata cables.

 

[VirtualLarry]

And? That's normal. Try doing a physical block-level benchmark, and see if your sequential read speeds match up?

 

[smitbret]

And? That's normal. Try doing a physical block-level benchmark, and see if your sequential read speeds match up?

My thought's, too.

 

[pcslookout]

That is perfectly normal.

 

[Elixer]

Depends where the data is located, and how fragmented it is. Best case data placement, and no fragmentation will indeed get you the max #'s they list.

 

[smitbret]

Depends where the data is located, and how fragmented it is. Best case data placement, and no fragmentation will indeed get you the max #'s they list.

.... as long as it is one, long, contiguous file.

 

[poohbear]

I think he's asking why he isn't getting the advertised 170mb/s -196mb/s transfer numbers instead of the 60mb/s he's getting. Are those transfer rates only if it's a single big file instead of the smaller files in a game folder?

 

[PingSpike]

Try running crystaldiskmark to give you an idea where to sniff next. But depending upon what kind of files you're transferring I agree with others that is probably normal.

 

[Anteaus]

If you see numbers that high it is usually related to caching. 60-80 MB/s is normal for writing to SATA3 hard drives. What is probably happening is that it is going to RAM first while it starts writing the files which will give a higher initial transfer speed. Once the cache fills up, the transfer speed will then settle to the actual speed.

 

[Ranulf]

Pretty normal depending on the files sizes like others mentioned. Sometimes its faster to do a steam backup and transfer over the backup files and reinstall from there or just re-download the games.

Given your drive size though, probably best to just direct copy over and wait. I backed up Jade Empire recently, a 6gb game that when just doing a raw copy from ssd to hdd, internal drives... 30 minutes or so. Tons of small files. Zipped/archived it went a lot faster. WoW, a 34gb install currently only takes about 20 minutes from a ssd to an external usb3 hdd. Lots of big files there.

 

[BFG10K]

65 MB/sec is far too low unless they're all small 4K files, which I doubt since most games try to have big "PACK" files where possible. Even my 5400RPM Green sustains well over 100 MB/sec for the entire copy duration of my ~440GB GB gaming library.

How were your copy speeds before you got the new drive? 90% of the time the problem is a background virus scanner, so disabled/exit anything like that and try again.

Also maybe the new drive is faulty? Try pairing an SSD with each HDD one at a time to try to isolate the problem. Also check your event viewer while copying to see if any I/O errors are logged.

Sometimes its faster to do a steam backup and transfer over the backup files and reinstall from there or just re-download the games.

Nonsense. There's nothing magical about Steam backup, it's just file copying.

As for downloading, no internet connection is going to come close to a modern mechanical HDD doing local I/O. The newest drives are capable of sustaining over 200MB/sec in real-world transfers.

 

[Ranulf]

Steam backups put the files into big chunks, usually 1gb, depending on if you choose "dvd" or "cd" backup. Thus the archive files transfer far faster than a raw backup of the install directory.

For me with my internet connection speeds (15 MB/s down), games that are 5gb or under it is often faster to re-download them than copy over the directory or reinstall from a steam archive, especially if the archive is on an external drive.

 

[TheRyuu]

As for downloading, no internet connection is going to come close to a modern mechanical HDD doing local I/O. The newest drives are capable of sustaining over 200MB/sec in real-world transfers.

Well at least at the start of the drive. It'll average out to be much slower than that over the entire drive but we have to assume his entire steam library probably isn't at the end of the drive.

It sounds like it's somehow only use half the available bandwidth. Even if your steam library was at the end of the drive it still seems a bit slow but maybe not impossible but one would assume that your entire steam library is not at the end of the drive. I'm not sure how your drives are setup but is it somehow possible that it's "sharing" the connection somehow since it seems like it's being capped by something such as the available bandwidth from one SATA cable.

 

[grimpr]

Always format your drives using 64k NTFS Cluster size and use a decent copy program to maximise your throughput and reliability when moving terrabytes of data, imho the best copier in the windows world and what i personally use and have tried many is FastCopy, http://ipmsg.org/tools/fastcopy.html.en

 

[TheRyuu]

Always format your drives using 64k NTFS Cluster size and use a decent copy program to maximise your throughput and reliability when moving terrabytes of data, imho the best copier in the windows world and what i personally use and have tried many is FastCopy, http://ipmsg.org/tools/fastcopy.html.en

The cluster size (or allocation unit size as Windows calls it) can depend on your use case. If you're unsure then leaving it at the default is fine. For a boot drive I would just leave it at the default. For a drive storing large amounts of media (e.g. music, videos) then you may want to set a larger one although again there shouldn't really be an issue using the default.

Also if using a RAID setup it may be beneficial to set the allocation unit size equal to the RAID stripe size so long as Windows can actually align itself to the stripe otherwise again the default just works.

Does changing it from the default really offer anything noteworthy? Also the default of 4096 bytes will align with the larger 4KiB sector size (advanced format) on modern large HDD's.

 

[grimpr]

Does changing it from the default really offer anything noteworthy? Also the default of 4096 bytes will align with the larger 4KiB sector size (advanced format) on modern large HDD's.

Im my opinion yes, i noticed higher tranfer rates, fewer dips in in throughput when copying around files, less fragmentation, faster defragmentation, overall better i/o which means a healthier drive which coped previously with 4K cluster size. There's lots of evidence in the benefits of 64K cluster size around the internet.

 

[tweakboy]

That is about right .. thx gl