Why have hard drive capacities slowed so much?

[exar333]

I think that about the time capacities hit 1 TB, they got about as big as most people will ever need. All my desktops at work have only 80 GB. Everyone stores their work on the file server anyway so a lot of space on individual machines would just be wasted.

For home users, for most people, 1 TB is as much as they will ever use and, personal opinion here, if you are running low, you are better off adding more 1 TB drives than you are going with a single bigger drive.

A final reason is that drive prices are so cheap that its hard for drive makers to justify larger capacities. For example I'm seeing a WD 2 TB Caviar Black for $85 on New Egg (out of stock at the moment). There is a 3 TB Hitachi for $120. Where is the incentive to bring out higher capacities? Wont they make more by selling you multiple smaller drives anyway?

More capacity is good, but the market for larger and larger drives are diminishing. For those not recording mass qualities of videos, a 2TB drive is HUGE. That is a lot of music, files, programs, and movies. I am not stupid and saying '2TB will be enough forever!' but we have stalled somewhat because the current drives are cheap and large for most users.

10 years ago, a 40GB drive was pretty standard. The OS took-up about 25% of the capacity. Now, a 2TB drive only uses 1-2% for the OS. Something to think about...

 

[exdeath]

Maybe because people realize that 2+ TB is useless when it takes 3 years to read/write/backup data to it at sub 100 MB/sec speeds. Seriously who uses "magnetic tape" to store shit anymore? Floppies might be slow at 1 K/sec but we only stored 1K files back then. What we are doing now is moving 10 GB files around at that same 1 K/sec figuratively speaking.

We've allowed the memory gap to become unsurmountable now. Processing power and data size has grown to ungodly levels with no end in sight and storage technology has remained relatively unchanged in over 50 years. Too much data and the inability to keep CPUs and GPUs fed with current outdated physics and storage technology. Hourglasses and progress bars are the prominent features of the GUI because of it. It's stopping the progress of computing technology dead in it's tracks.

If you have to wait 30 mins to uninstall and reinstall a bloated email client, there is no way you are recalling and evaluating gigabytes of data from non volatile storage in the blink of an eye for things like human-like computer vision and AI.

The next revolution in computing absolutely HAS to come from a breakthrough in non volatile non mechanical universal data storage. Screw hard drives. NAND Flash too needs to be short lived too. Let them die. There is no excuse for processing data at MB/sec and KB/sec speeds in this age.

Bring on the high density high capacity STT-MRAM for main memory already and eliminate the need for HDDs/SDDs and eliminate the concept of "loading".

 

[taltamir]

I think that about the time capacities hit 1 TB, they got about as big as most people will ever need.

Well, need is a funny word. It is nice having my entire video library as 15GB bluray rips on my fileserver... This has really pushed up my space usage from having 5GB bluray rips which in turn was a little higher than my 2GB each DVD rips which was higher then earlier 0.5GB DVD rips.

I also keep local copies of every game I buy online on the server for convenience and in case servers go down (you can do offline installation with most direct download services)

Of course this can only go so far. Going from 15GB rip to 45GB uncompressed bluray image isn't doing much if anything for quality.. and then where will I go? Well, probably some next gen 3D thing at an even more absurd resolution...

But my mother would never break the 100GB barrier as she has neither.

 

[imagoon]

Seriously who uses "magnetic tape" to store shit anymore?

Just about every business entity with more than 100ish or so people. Modern tape can keep up and exceed HDD speeds sequentially. Obviously they fail at random but that is not the point for snapshot backups.

 

[Jeff7181]

Just about every business entity with more than 100ish or so people. Modern tape can keep up and exceed HDD speeds sequentially. Obviously they fail at random but that is not the point for snapshot backups.

Yep. Modern tape is quite fast and requires multi-Gbit data streams to keep it from emptying the buffer constantly while writing to the tape.

 

[videopho]

Here is for your 2.2 TB LIMITATION bed time reading tonight, lol...

What is the 2.2TB limitation?

Users upgrading a computer system with a
larger hard disk drive may have problems using
more than 2.2 terabytes (TB) of disk space.
This limit arises from the maximum size of the
master boot record (MBR) partitioning method
used by most personal computers.
This document explains the issue and
discusses long term solutions based on GPT
and Unified EFI (UEFI) specifications.

Why does this limitation exist?

This limitation dates back to the 1980s and the
original IBM PC. This introduced the master
boot record (MBR) partitioning scheme to
describe hard disk partitions.
BIOS systems with MBR disks use 32-bit values
to describe the starting offset and length of a
partition. Due to this size limit, MBR allows a
maximum disk size of approximately 2.2 TB and
a maximum of four primary partitions.
How does this affect today’s computers?
To meet the demands of modern computer
users, hard disk drive manufacturers will
produce disks in excess of 2 TB in 2010. Users
upgrading a computer system by adding or
replacing an existing hard disk drive with a
larger drive may encounter problems using
more than 2.2 TB of disk space.

How will BIOS and OS address this limit?

The UEFI specification defines a new model for
the interface between personal-computer
operating systems and platform firmware,
updating BIOS interfaces such as MBR.
UEFI supports the GUID Partition Table (GPT),
a more flexible partitioning scheme. GPT disks
use 64-bit values to describe partitions, allowing
larger partitions. GPT also fixes other issues
related to MBR (data integrity, backup tables,
maximum number of partitions, &#8230.
Using 64-bit values, GPT can handle disks of up
to 9.4 x 1021 bytes or 9.4 zettabytes (ZB).
Will the OS boot to a partition over 2.2TB?
Only operating systems supporting UEFI and
GPT are expected to boot from partitions larger
than 2.2TB. This also requires the underlying
firmware to implement UEFI.
Microsoft and Linux already support UEFI in
newer OS revisions. Microsoft has published
several documents related to UEFI & GPT:
• Windows and GPT FAQ [Microsoft WHDC]

BTW, most current mobo mfgs now have added driver to increase the 2.2 tb barrier.

 

[videopho]

For multimedia storage (BR, DVD etc.), I recommend using external HDD where they can be found relatively inexpensive or you can buy external enclosure and build your own storage sizes.
I have done both myself: build and buy.
My current project is I already bought a 6tb RAID0 enclosure for $50 and now just waiting for 3TB drive prices to drop some more then there I will have my 6tb storage.

 

[videopho]

Lol statements like this are almost always patently false. We were told the same thing when we got our first 40MB hard drive. For the near term your right though. I could kill a 1TB drive easy with a selection of blu-ray movies and a few games. I think once digital distribution of media gets in full swing hard drive space will shrink rapidly. Mass physical mediums such as blu-ray, dvd or cds are already on borrowed time. It might take 5-10 years before high speed internet because fast enough and widespread enough to kill them, but it is inevitable. Why? Control

+1
For my HTPC, the current library (movies, music videos and all other collections) is now sitting at 7.5Tb and growing, soon will be another additional 6tb.

 

[taltamir]

Here is for your 2.2 TB LIMITATION bed time reading tonight, lol...

http://www.rodsbooks.com/gdisk/booting.html
http://t13.org./documents/UploadedDocuments/docs2010/e09127r3-EDD-4_Hybrid_MBR_boot_code_annex.pdf

Behold, booting a GPT partition is perfectly possible with BIOS.
Windows is the ONLY OS that requires UEFI to boot from GPT. Every other OS has the ability to boot GPT from a bios

 

[Nothinman]

Your rant doesn't make sense.... Windows supports EFI/UEFI. I have a couple of servers running it and did some testing with Windows 7 which also works. Your complaints about XP not supporting it is about the same as me whining that Debian REX doesn't have USB3 support....

Hardly. Last I checked, XP still had a larger install base than Win7 so it makes perfect sense that hardware manufactures wouldn't be in a tizzy to produce hardware that can't easily work in it. And your analogy is extremely broken, if for no other reason than that Rex was released a full 5 years before XP. And I bet I'd can get USB3 working on Rex with less work than you could get XP to install and boot from a EFI only motherboard.

 

[MustangSVT]

because porn files are not getting larger.

 

[Anteaus]

I really don't know what the big deal is. Linux had EFI before Windows, but it's not like there was a imperative to have it on the x86 side at the time. MS added EFI support after XP, but again it's not like there was a ready supply of EFI motherboards at the time. One could argue that had Windows adopted EFI sooner (XP <), EFI motherboards might have hit the market earlier but would harddrive capacities have increased more rapidly? That's a hard one to sell.

 

[Voo]

Hardly. Last I checked, XP still had a larger install base than Win7 so it makes perfect sense that hardware manufactures wouldn't be in a tizzy to produce hardware that can't easily work in it. And your analogy is extremely broken, if for no other reason than that Rex was released a full 5 years before XP. And I bet I'd can get USB3 working on Rex with less work than you could get XP to install and boot from a EFI only motherboard.

So I can get support for UEFI boots from Redhat or some other linux vendor for a 10year old distribution? That'd be quite surprising (but who knows, I haven't looked it up) - or don't we have to apply the same standards to all?

 

[Anubis]

Hardly. Last I checked, XP still had a larger install base than Win7 so it makes perfect sense that hardware manufactures wouldn't be in a tizzy to produce hardware that can't easily work in it. And your analogy is extremely broken, if for no other reason than that Rex was released a full 5 years before XP. And I bet I'd can get USB3 working on Rex with less work than you could get XP to install and boot from a EFI only motherboard.

Win 7 install base is larger, it just passed XP

http://www.dailytech.com/Windows+7+...t+Two+Years+to+Become+Top+OS/article23016.htm

 

[taltamir]

because porn files are not getting larger.

that is not true. porn was the first industry to adopt HD.
I remember when porn was 280p resolution in rm format, a 2 hour morive was 16mb.
A full HD bluray rip movie at high quality encode could easily take 5GB even with x264

 

[Anteaus]

This is a side bar, so feel free to disregard as it doesn't really pertain directly to the topic at hand, but does anyone else get annoyed with the acronym HD is used to describe video? After all, as knowledgable computer users we all know that what the layman calls HD, we know is just whatever the highest resolution used by mainstream television. 1280X720 was called HD at one time. Does that mean it is no longer considered HD now that 1920X1080 is here? How can something be "high definition" one second and not be the next? Based on resolution criteria, I was operating in "HD" the first time I went to 1280X1024, even if it wasn't "widescreen". Sorry for the rant. Please continue.

Oh, and yes you can download porn at 1920X1080 now.

 

[taltamir]

720p was called "Half-HD" and 1080p was called HD from the very get go. But a lot of people selling 720p products "forgot" to mention the "half" part.
But yes as a computer user it is annoying. I was using HD resolution since 2001 IIRC... even AMD went and renamed their cards the "RadeonHD" when I was playing HD resolution with radeons for many years before.

 

[Voo]

As long as I know what is meant by it, I don't care - seems like useless nitpicking.

Also give me 720p every day over 1080i..

 

[Jeff7]

What happens with the read error rate problem?
I saw something about that awhile back that, given the statistical chance of an unrecoverable read error occurring versus the size of a large drive, there's a reasonable chance of encountering such an error if you were to try to read back everything on a full drive. This could then trigger a warning or array rebuild on a RAID setup, or some other undesirable thing.
Are there improvements being made for this read error rate, or new compensation methods, or what?

With a RAID 1 setup with a pair of drives, if one drive has an "I done gone and screwed up" moment on a sector, will it automatically and transparently try to retrieve the data from the other one?

This sort of thing.

2 hundred million sectors is about 12 terabytes. When a drive fails in a 7 drive, 2 TB SATA disk RAID 5, you&#8217;ll have 6 remaining 2 TB drives. As the RAID controller is reconstructing the data it is very likely it will see an URE. At that point the RAID reconstruction stops.

Lol statements like this are almost always patently false. We were told the same thing when we got our first 40MB hard drive. For the near term your right though. I could kill a 1TB drive easy with a selection of blu-ray movies and a few games. I think once digital distribution of media gets in full swing hard drive space will shrink rapidly. Mass physical mediums such as blu-ray, dvd or cds are already on borrowed time. It might take 5-10 years before high speed internet because fast enough and widespread enough to kill them, but it is inevitable. Why? Control

I would love to rip every DVD I own and have them right on the computer, in original format, instead of converting to some AVI format to cut the filesize in half, which of course adds extra work, and a little bit of quality loss.

 

[taltamir]

What happens with the read error rate problem?

4K sectors http://www.anandtech.com/show/2888

I saw something about that awhile back that, given the statistical chance of an unrecoverable read error occurring versus the size of a large drive, there's a reasonable chance of encountering such an error if you were to try to read back everything on a full drive. This could then trigger a warning or array rebuild on a RAID setup

Actually, with standard RAID it results in a failed rebuild. The rebuild process simply aborts midway and you lose all your data (unless it was backed up).
This hasn't changed, but there are non standard RAID-like implementations that do things better. For example ZFS has RAIDz which is RAID5 only without the stupid.

I would love to rip every DVD I own and have them right on the computer, in original format, instead of converting to some AVI format to cut the filesize in half, which of course adds extra work, and a little bit of quality loss.

I find a 2GB bluray rip (compressed from the natively ~40GB original) to be vastly superior to a 4.7GB untouched DVD image.

 

[imagoon]

Actually, with standard RAID it results in a failed rebuild. The rebuild process simply aborts midway and you lose all your data (unless it was backed up).

Only on crappy controllers / bad implementations. Smart ones will log the error, relocate the sector on the media and keep on rebuilding. The sector of course is lost, but that is no different than if a sector goes URE on a single disk system anymore.

 

[kmmatney]

I have a pretty huge movie/music collection, and have 6 com[puters backed up on my WHS, and I still have a ton of disk space left. I'm running 2 TB drives plus a 200 GB boot drive and a 320 GB drive (thrown in for good measure) on my WHS 2007, with drive extender.

I'd prefer increasingly the reliability on the existing high capacity drives, over making even bigger ones. I'm sure they will get there in time, though. I think that capacity has met demand for now.

 

[Jeff7]

4K sectors http://www.anandtech.com/show/2888

Well that's good.
How long then until we need 8K sectors? 7 years?

Actually, with standard RAID it results in a failed rebuild. The rebuild process simply aborts midway and you lose all your data (unless it was backed up).
This hasn't changed, but there are non standard RAID-like implementations that do things better. For example ZFS has RAIDz which is RAID5 only without the stupid.

Oh, well...that's fine too.

I find a 2GB bluray rip (compressed from the natively ~40GB original) to be vastly superior to a 4.7GB untouched DVD image.

I have yet to make the move to high-def anything.
Well, you might count my monitor, which is 1080 pixels high. :awe:
Going by that though, my 21" Trinitron-tube CRT was high-def too, as it could easily handle 1600x1200.

 

[Puppies04]

Even if they started shipping 10 tb drives tomorrow who would buy them, maybe 0.1&#37; of personal comuter users so the markup would have to be astronomical (maybe $1000) a drive. This is the reason they don't exist, 99.9% of consumers dont need/want them

 

[taltamir]

Even if they started shipping 10 tb drives tomorrow who would buy them

Google, amazon, yahoo, and other companies that need massive servers.