How Many HDDs/RAID Arrays Is Too Many??

[animeguru]

I didn't think this was really a "General Hardware" question, so I pose it to you guys.

I'm building up a little dualy server for myself and I have a question.

Well, first off, the specs...

Intel L440GX+
Dual P3 550 (100FSB)
512 MB PC100 Registered ECC RAM (possibly bumping to 1GB or more)
One U160 9.1 GB HDD from on board SCSI for OS
LSI MegaRAID 1600 SCSI RAID Card (Dual channel)
- 9 internal U160 9.1 GB drives to be in one RAID 5 array
- 8 external U160 36.7 GB drives in one RAID 5 array
Adaptec 3944AUWD SCSI Card (Dual channel)
- with one Sun A1000 (Hardware RAID array) full of 12 36 GB SCSI HDDs
- up to two Sun D1000 (JBOD) arrays chained off of the A1000 also full of drives (another 20 HDDs)

(Link if that was too confusing)
http://www.anandtech.com/mysystemrig.html?rigid=25363


So, I'm starting to think that it'll be total overkill for the one server. I mean, 4 external arrays chained off of one lowly dual 550 may be too much, 5 total RAID arrays, possibly more.

Will I be totally crushing the server with so many arrays / HDDs?? Would it be adviseable to build a second server to handle some of the arrays or would one more powerful server be a better idea.

Also, what is the maximum number of disks recommended for a RAID 5 array?? As far as I know, it's only limited by the available channels and SCSI IDs. However, the Sun A1000 being a hardware RAID box, I think it can control the two D1000's and utilize them for aditional RAID storage space (I'm not terribly familiar with Sun and I got these for free, so I'm playing with them). So, there may be as many as 24 HDDs in one RAID 5 array (depends on how many of the same size drives I can pull together).

Any ideas and coments are totally welcome!!

 

[Matthias99]

Originally posted by: animeguru
I didn't think this was really a "General Hardware" question, so I pose it to you guys.

I'm building up a little dualy server for myself and I have a question.

So, I'm starting to think that it'll be total overkill for the one server. I mean, 4 external arrays chained off of one lowly dual 550 may be too much, 5 total RAID arrays, possibly more.

Will I be totally crushing the server with so many arrays / HDDs?? Would it be adviseable to build a second server to handle some of the arrays or would one more powerful server be a better idea.

Hard to say. Hardware RAID controllers offload most of the I/O load from the CPU, so it may not be much of a restriction depending on how many I/Os per second you're trying to do. Your bigger bottleneck may be getting data in and out of the server and onto the hosts, since you'll have trouble pushing both network and disk traffic over the PCI bus simultaneously. I mean, you're talking a disk array that can probably push hundreds of MBs/sec, if not more. You just won't have bandwidth to feed it.

Also, what is the maximum number of disks recommended for a RAID 5 array?? As far as I know, it's only limited by the available channels and SCSI IDs.

I don't think I've seen controllers that will let you put more than 8 drives in a single RAID5 (unless it's being done via software). You have to recalculate parity across all the disks when you write, so adding more disks starts to bog you down (since you may have to wait for multiple disks to finish reads before they can write the parity data back). Also, a RAID5 can only tolerate the loss of one disk, so there are safety concerns with involving more disks than that. With 8 disks it's unlikely that a second disk will fail before you can swap out the first busted one. With 20 or 30 disks this becomes a distinct possibility.

If you need a bigger virtual disk than 7 times a single drive's capacity, you can make two RAID5s and RAID0 them together (RAID5+0). Even if the controller doesn't support it, you should be able to do *that* through the OS.

 

[jagec]

so tell me

where'd you get that much sweet hardware for free?

you been stealing from the .coms?

I think for your number of drives RAID 50 would be best (pretty much what matthias suggested). Chain together what you can, and just JBOD the odd drives out.

What are you going to do with almost 1 TB of storage?

 

[animeguru]

Originally posted by: jagec
so tell me

where'd you get that much sweet hardware for free?

you been stealing from the .coms?

I think for your number of drives RAID 50 would be best (pretty much what matthias suggested). Chain together what you can, and just JBOD the odd drives out.

What are you going to do with almost 1 TB of storage?

:Q ... I would never do something like that...

As to it's use, well, I have a lot of por...

It'll actually be an audio/video server for my house. I'm ripping my DVD collection to disk for streaming across my network. I'm using DVDShrink to take the size down a bit, but each one is still around 5GB, so it's a lot of HDD space. Plus, I have a rather large number of MP3s and plenty more CDs that need to be converted.

I may end up utilizing a second server for the 3 sun arrays since they'll probably be the most CPU intensive. I haven't installed the utility yet to configure and run them, so I'm not sure. I suppose that I'll just have to try it out and see.

don't think I've seen controllers that will let you put more than 8 drives in a single RAID5 (unless it's being done via software). You have to recalculate parity across all the disks when you write, so adding more disks starts to bog you down (since you may have to wait for multiple disks to finish reads before they can write the parity data back). Also, a RAID5 can only tolerate the loss of one disk, so there are safety concerns with involving more disks than that. With 8 disks it's unlikely that a second disk will fail before you can swap out the first busted one. With 20 or 30 disks this becomes a distinct possibility.

Yeah, I guess you're right on that one. I know that the LSI controller will allow me to put as many drives into an array as it can hold (I had one that was 10 disks), but I'll probably try to limit the total number of disks in each array for the reason you said. Honestly, the efficiency (percentage of total space used for parity) of the drive space doesn't really drop too much with an increased number of disks.

 

[Matthias99]

Originally posted by: animeguru

don't think I've seen controllers that will let you put more than 8 drives in a single RAID5 (unless it's being done via software). You have to recalculate parity across all the disks when you write, so adding more disks starts to bog you down (since you may have to wait for multiple disks to finish reads before they can write the parity data back). Also, a RAID5 can only tolerate the loss of one disk, so there are safety concerns with involving more disks than that. With 8 disks it's unlikely that a second disk will fail before you can swap out the first busted one. With 20 or 30 disks this becomes a distinct possibility.

Yeah, I guess you're right on that one. I know that the LSI controller will allow me to put as many drives into an array as it can hold (I had one that was 10 disks), but I'll probably try to limit the total number of disks in each array for the reason you said. Honestly, the efficiency (percentage of total space used for parity) of the drive space doesn't really drop too much with an increased number of disks.

The 'efficiency' actually gets *better* with an increased number of disks, as there is less parity data relative to actual data. The normal configurations you see are 3:1 (or "3+1") and 7:1 (or "7+1") -- that is, 3 disks of data and 1 of parity (75% usable space), or 7 disks of data and 1 of parity (87.5% usable space). If you set up an 11:1 RAID5 with 12 disks, you'd have 91.7% usable space. Note that with RAID5, the parity is striped across all the disks, so there is no dedicated parity drive (as in, I think, RAID3).

But while your space utilization gets better, having to write parity data to every disk for every write starts to seriously bottleneck performance (unless you have a VERY read-heavy I/O load), and you start noticeably increasing the odds that you'll have two disks in the same array go bad within a short timeframe. In most situations, it would make more sense to build several smaller RAID5 arrays and RAID0 them together if you need a very large virtual disk.

 

[JHutch]

As someone who has done a video server himself, I'd have to recommend looking into one of the MPEG4 codecs. XViD is my personal favorite. You can get the movie under 1GB pretty much on all examples (a few exceptions) and the quality is virtually identical to the original DVD when played on a TV screen.

JHutch

PS My server is at around 600GB right now, with plans to increase probably later this summer, early fall.