Large storage solution - Software Raid 5?

[dorfma05]

Hello all. I am looking to store a large amount of media: movies, tv, music, etc.

Speed isn't a huge concern as I don't think this will impact watching a movie, etc. (Correct me if I'm wrong.)

I backup everything I can't afford to lose already so that really has no bearing on this.

All I want is a large disk solution, 5-10TB to store all of these files. The only thing I'm scared of from a redundancy standpoint is the mechanical failure of a drive.

Is software raid 5 a good way to protect against this? If so, what's the best software to use? Also, can a software raid 5 array be expanded in the future? (If not, I could always make a second array down the line.)

I'm pretty much roughly looking to do the software equivalent of what the Drobo does without getting locked into proprietary raid hardware which could fail. If the software solution isn't expandable like the Drobo, I can deal with that.

Thanks all.

 

[Mark R]

Most software RAID is expandable - but lacks the flexibility of the drobo.

Some implementations will require that you replace each individual drive by a larger capacity drive. E.g. you start with 4 1TB drives. You replace one with a 2 TB drive, wait for the RAID to recover. Then repeat with the next, until all 4 have been replaced. Once the RAID recovers after the 4th drive has been installed, the new capacity comes online.

This isn't ideal from a reliability perspective - as you are deliberately breaking the RAID 4 times. And problem with regeneration (e.g. a previously undetected bad sector), and you are screwed. However, from a compatibility perspective, pretty much any software RAID can be extended in this way.

Some RAID systems (notably Linux) will allow you to add an extra drive. E.g. you have a 4 drive RAID 5. You can then add a 5th identical drive, and it can be merged into the RAID, to give a 5 drive RAID 5. There is no breaking of redundancy at any time (although, the complexity of dealing with a partially migrated RAID does bring the risk of some obscure software bug biting your ass).

This 2nd mode of upgrade is much less well supported (as generally enterprise users don't need this - they just add a new array), so it's really only linux that offers it. However, if you use one of the common-or-garden NAS units (like synology or QNAP), then, as they are based on a standard linux system, they will offer this functionality.

 

[Binky]

Try these links:

http://snapraid.sourceforge.net/compare.html

http://www.avsforum.com/avs-vb/showthread.php?p=13573788#post13573788

The AVS home theater computer forums have a decent amount of discussion about media servers without focusing too much on any one setup. Check there.

 

[dorfma05]

Thanks all for the responses. That snapraid thing seems like it's exactly what I'd want.

 

[greenhawk]

raid 5 can help protect it. but the down side is cpu usage (can get high if a drive drops or is being rebuilt. A issue if using the cpu to decode and display/stream media at the same time if looking for a low power setup.

Another issue is that even to get a short video, raid 5 will require all drives to spin up to be accessed. When getting to 6+ drives (needed for 10TB raid 5, using 2tb drives), having that many drives might be a issue if in the same room watching the show. and of course power if that is a consern.

expantion can be iffy personally depending on setup, but hardward raid 5 is expencive.

Personally, after playing with raid 5 (hybrid setups and one software), I would be looking at a a set of raid 1 arrays. Needs more drives, but only 2 drives needed to spin up at a time, very low CPU over head and quieter overall when single video streaming. But if you have the data backed up, then using NTFS junctions can be better again (with no raid).

 

[Binky]

If you want something very simple, look at windows home server with drivepool. This will cost around $50-$70, but it can provide other functions as well (backup, remote access, DLNA server). If you're a windows user, the interface and configuration screens will be very familiar.

 

[jwilliams4200]

Thanks all for the responses. That snapraid thing seems like it's exactly what I'd want.

I've been using snapraid on my linux media server for more than 6 months now, and it works well and is easy to setup and use.

Snapshot RAID is a much better choice for a media server than distributed parity RAID :

o works with HDDs of different sizes

o no data migration necessary since it just uses drives formatted with your normal filesystem

o if you lose more drives than you have parity, you only lose the data from the dead drives

o during movie playback, only one drive needs to spin up

o more efficient use of total drive capacity than 1-to-1 mirroring

 

[VirtualLarry]

Snapshot RAID sounds a bit like unRAID.

I was planning on using unRAID with my 20TB (10x2TB) server. Still waiting to save some money for license keys for the Pro version.

edit: http://www.lime-technology.com

 

[jwilliams4200]

Some similarities, but snapraid beats unraid:

o snapraid is free, GPL, open-source

o snapraid does not require data migration (unraid requires reiserfs fileystem)

o snapraid supports dual-parity, but unraid only supports single-parity

o snapraid maintains checksums on all your data (unraid does not)

o snapraid can run on any OS (you aren't locked in to a stripped-down linux distro like unraid)


http://snapraid.sourceforge.net/compare.html

 

[beginner99]

1 question about snapRAID:

So I basically have 2 currently completely independent drives (no RAID whatsoever) I want to use with snapRAID. I could now add a 3 one to use as parity drive and run

parity H:\par\parity.par
content E:\array\content.lst
content F:\array\content.lst
disk d1 F:\array\
disk d2 G:\array\

and I'm ready to go?

Is it required to have the content file on a disk that is nor part of the array?

 

[jwilliams4200]

Yes, that looks fine (except for the smilies, you should use

code tags

)

No, the content file can go on a data disk.

For future snapraid questions, I suggest the snapraid help forum:

https://sourceforge.net/projects/snapraid/forums/forum/1677233

 

[Vegemeister]

ZFS on FreeNAS or FreeBSD. Use RAID Z1 or Z2 (like RAID 6). It has checksumming and you can schedule scrubbing to go through and verify the checksums. Btrfs has similar features on Linux, but I wouldn't trust it yet.

The speed will not be a problem. Any modern CPU can do parity calculations in its sleep.

 

[Cerb]

Yes, that looks fine (except for the smilies, you should use

code tags

)

No need. The 3rd checkbox in Miscellaneous Options, in the Additional Options panel, disables them. A quick advanced mode edit, just checking that box, can remove them.

 

[taltamir]

The term "software" is a big misleading. All RAID is software, sometimes there is a firmware component involved (but firmware IS software too)

You basically have 3 choices:
1. OS based software RAID: This is where the RAID is done entirely in driver. This is the most reliable, but kinda slow.
2. Cheap controller based RAID: This is what mobo controllers and 30$ controllers do, they hook into bios, load firmware before the OS does, and use the system's CPU and RAM to perform their calculations. This is slowest and most unreliable RAID you can get.
3. Expensive controller based RAID ($300+): These controllers have their own processor and RAM and do their own parity calculations locally. Very fast, reliability varies but usually less reliable then OS based.

I recommend you go with OS based and use nas...
FreeNAS is a good place to start, use ZFS.

 

[Silenus]

Personally I'm a fan of ZFS. I have two storage machines I run at work based on OpenIndiana + Napp-IT web gui. Fantastic machines with enterprise class features. FreeNAS is a simpler/easier to set up but still gives you the benefits of ZFS (copy-on-write, full end to end checksumming, automatic scrubbing, instant snapshots, and more).

Regardless of what you do I would most definitely NOT do RAID 5, or RAID Z1, ect. Single drive parity is just not enough in my opinion onces you get past a few TB or larger than 1TB drives. For the amount of data you are talking about I'd go RAID 6 (or ZFS RAID Z2). If your super paranoid ZFS allows even RAID Z3 (triple drive parity!).

 

[taltamir]

Personally I'm a fan of ZFS. I have two storage machines I run at work based on OpenIndiana + Napp-IT web gui. Fantastic machines with enterprise class features. FreeNAS is a simpler/easier to set up but still gives you the benefits of ZFS (copy-on-write, full end to end checksumming, automatic scrubbing, instant snapshots, and more).

Regardless of what you do I would most definitely NOT do RAID 5, or RAID Z1, ect. Single drive parity is just not enough in my opinion onces you get past a few TB or larger than 1TB drives. For the amount of data you are talking about I'd go RAID 6 (or ZFS RAID Z2). If your super paranoid ZFS allows even RAID Z3 (triple drive parity!).

I am currently running RAIDZ2 (RAID6 equivalent but without the write hole) and I am regretting not using multiple RAID1 arrays. It is difficult to upgrade capacity on the same machine with so many drives tied up per array.

 

[jwilliams4200]

No need. The 3rd checkbox in Miscellaneous Options, in the Additional Options panel, disables them. A quick advanced mode edit, just checking that box, can remove them.

The "disable smilies in text" option does not show up if, in your User CP, your "edit options"/"miscellaneous options"/"message editor interface" is set to "basic editor - a simple text box", which I believe is the default setting. If you have "standard editor - extra formatting controls", then you will have the "disable smilies in text" option that you mentioned.

Anyway, I think it is better to just enclose things like configuration files in

[CODE]code tags[/CODE]

By the way, I surrounded the above in HTML tags so that the CODE tags did not get interpreted. But I am referring to CODE tags, not HTML tags.

 

[jwilliams4200]

...I am regretting not using multiple RAID1 arrays. It is difficult to upgrade capacity on the same machine with so many drives tied up per array.

This is another reason why ZFS is a poor choice for a media server, in addition to the other reasons I listed in post #7

ZFS is an even worse choice for a media server than most other distributed-parity RAIDs, since most others (mdadm, most hardware RAID) allow online capacity expansion (OCE). With ZFS, the most you can do is add another RAID vdev to your existing zpool. But for RAIDZ2, the additional vdev must use at least 4 drives, and even then no one uses 4-drive RAIDZ2 vdevs, since that is only 50% storage efficiency, same as a mirror. So, assume at least 5 drives. You can only expand your ZFS RAIDZ2 pool by 5 or more drives at once. Not good for a media server.

As I said earlier, the best choice for a media server is snapshot RAID. And the best free snapshot raid is SnapRAID. If you are willing to pay for it, FlexRAID also offers snapshot RAID, but I prefer SnapRAID since it is simpler.

 

[jwilliams4200]

The term "software" is a big misleading. All RAID is software...

Wrong. All RAID is not software. Hardware RAID cards have a ROC (RAID on chip) that does the required RAID computations in hardware.

1. OS based software RAID: This is where the RAID is done entirely in driver. This is the most reliable, but kinda slow.

Not all software RAID is slow. Linux software RAID, mdadm, is just as fast as hardware RAID, i.e., the limiting factor is the read and write speed of the underlying drives. It is not difficult to configure an mdadm distributed-parity RAID that runs at 1000 - 1500 MB/s for sequential reads and writes.

 

[taltamir]

Wrong. All RAID is not software. Hardware RAID cards have a ROC (RAID on chip) that does the required RAID computations in hardware.

Do read the rest of the post where I clarify before you start yelling.

Not all software RAID is slow. Linux software RAID, mdadm, is just as fast as hardware RAID, i.e., the limiting factor is the read and write speed of the underlying drives. It is not difficult to configure an mdadm distributed-parity RAID that runs at 1000 - 1500 MB/s for sequential reads and writes.

I was speaking in general terms. But ok, fine, good exception.

 

[jwilliams4200]

Do read the rest of the post where I clarify before you start yelling.

I did read the rest of your post, and you had other minor errors which I did not bother correcting.

 

[taltamir]

ZFS is an even worse choice for a media server than most other distributed-parity RAIDs, since most others (mdadm, most hardware RAID) allow online capacity expansion (OCE). With ZFS, the most you can do is add another RAID vdev to your existing zpool. But for RAIDZ2, the additional vdev must use at least 4 drives, and even then no one uses 4-drive RAIDZ2 vdevs, since that is only 50% storage efficiency, same as a mirror. So, assume at least 5 drives. You can only expand your ZFS RAIDZ2 pool by 5 or more drives at once. Not good for a media server.

1. You can do online capacity expansion on ZFS. Simply swap each drive in turn with a bigger one and then perform a resliver. It means I need to upgrade 5 drives at a time though since this is the size of my array.
2. You can add a RAID1 2 disk vdev to an existing RAID6 5 disk array, for example. They vdevs don't have to match. Heck you could add individual disks not in any form of RAID or even RAID0 if you were crazy.

I am saying that I should have done 1 vdevs of 2 disk RAID1, then added a 2nd vdev, then a 3rd, then performed upgrade cycles on oldest RAID1 vdev. Allowing me to upgrade the array 2 drives at a time to save on cost and not buy unneeded amounts of space. (since price keeps on going down and capacity going up)

 

[jwilliams4200]

1. You can do online capacity expansion on ZFS. Simply swap each drive in turn with a bigger one and then perform a resliver. It means I need to upgrade 5 drives at a time though since this is the size of my array.
2. You can add a RAID1 2 disk vdev to an existing RAID6 5 disk array, for example. They vdevs don't have to match. Heck you could add individual disks not in any form of RAID or even RAID0 if you were crazy.

I am saying that I should have done 1 vdevs of 2 disk RAID1, then added a 2nd vdev, then a 3rd, then performed upgrade cycles on oldest RAID1 vdev.

1. Who would be crazy enough to waste their time doing that? And who wants to upgrade 5 drives at a time?

2. That would be wasteful, since you are only getting 50% efficiency for the added capacity.

It was clear what you were saying. But it does not change the fact that ZFS is a terrible choice for a media server.

 

[taltamir]

1. Who would be crazy enough to waste their time doing that? And who wants to upgrade 5 drives at a time?

Not me, that is why I am not doing it. But you said IMPOSSIBLE not "inconvenient"

2. That would be wasteful, since you are only getting 50% efficiency for the added capacity.

Huh? how do you figure?

It was clear what you were saying. But it does not change the fact that ZFS is a terrible choice for a media server.

If you are using a 5 or 6 drive raid6 then its not ideal.
But what about my 3 vdev RAID1 pairs suggestion? that isn't terrible choice at all.

 

[jwilliams4200]

Not me, that is why I am not doing it. But you said IMPOSSIBLE not "inconvenient"

The process you described is NOT online capacity expansion, which refers to the capability to add one or more drives to an existing RAID device, and have the RAID expanded to the new drive(s).