Storage Server RAID Recommendations

[hennessy1]

I am planning on putting together a storage server primarily used for images of PC's. I wanted your opinions on the best way to go about setting up one or more RAID sets. I want to go with RAID 6 and possibly RAID 60. I will list the hardware at my disposal for this so you can point me in the right direction or offer suggestions. Thank you for any help.

Motherboard-Asus P8C WS

CPU-Xeon E3-1275 V2

Memory-Kingston 32GB (4 x 8GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1600

RAID Card-MegaRAID SAS 9280-24i4e

RAID Hard Drives-WD 3TB Red x24
(I have 24 on hand right now but I do not need that much storage to start off so it is not necessary to use them all for this just wanted to add that they are there.)

My initial plan was for a massive RAID 60 with all of them. I now think doing 2 independent RAID 6 would be best and have one as the primary use and the second as the backup set. I chose RAID 6 because of the 2 parity disks instead of RAID 5 single parity disk. I could also do 2 RAID 6 arrays with 10 each and then have 4 for spare.

 

[smitbret]

I would go with 2 independent systems, then a single catastrophic event doesn't wipe you out clean.

I since it sounds like archival storage and large file sizes will be the norm, I would suggest a ZFS RAIDz2 instead of hardware RAID 6. You'll get better error correction and protection against thinks like bit flopping and data rot.

 

[Phynaz]

Is this for your job?

You don't build your own for your job unless you want to be out of that job.

 

[smitbret]

I would go with 2 independent systems, then a single catastrophic event doesn't wipe you out clean.

I since it sounds like archival storage and large file sizes will be the norm, I would suggest a ZFS RAIDz2 instead of hardware RAID 6. You'll get better error correction and protection against thinks like bit flopping and data rot.

 

[CoolRunnings]

Another vote for going with ZFS here. The data security would be well worth it!

 

[Ayah]

You're going to be crippled with that memory to storage ratio though.
But I am a ZFS fan.

 

[smitbret]

You're going to be crippled with that memory to storage ratio though.
But I am a ZFS fan.

Not really. If he's going to be splitting up the number of drives at 10 x 3TB each, then 32 GB of memory will be fine. Even if he splits the memory, most features of ZFS will run fine with 16GB for 27TB of storage.

 

[hennessy1]

I tried looking for higher capacity sticks but the ones listed are the highest I could find with ECC and unbuffered memory. I mean I could use non ecc but with this size RAID I didn't want to have that in there.

 

[nanaki333]

I would go with 2 independent systems, then a single catastrophic event doesn't wipe you out clean.

I since it sounds like archival storage and large file sizes will be the norm, I would suggest a ZFS RAIDz2 instead of hardware RAID 6. You'll get better error correction and protection against thinks like bit flopping and data rot.

i had data rot once. the doctor just gave me some cream and it went away!

 

[Ayah]

Not really. If he's going to be splitting up the number of drives at 10 x 3TB each, then 32 GB of memory will be fine. Even if he splits the memory, most features of ZFS will run fine with 16GB for 27TB of storage.

I don't see him going with division into 10 drive vdevs with a 24 drive total, either into 12 or 8. My server isn't that large but 16GB to push data to 24 * 3TB drives? There are times when you'll push data simultaneously to every vdev.. Not ideal, but people aren't ideal.

Also 32GB is the maximum for lga1155. I've never heard of anyone recommending <512MB RAM / TB data.

As well, dump the ROC and go with a dumb HBA if you go ZFS. If you do, I'd put that money into moving to a LGA2011 platform based on a X9SRH-7TF. This will give you X540 for 10G copper, as well as the ability to load a ton of large RDIMMs.

 

[smitbret]

He said he was considering 2 independent RAID 6 setups, one for primary and the other for backup. Said he had 24 x 3TB drives and would most likely hold 4 back as spares. Since only 1 array would be accessed at a time, under a RAIDz2, he'll have a 24TB pool with 10 HDDs.

 

[hennessy1]

I am not against using zfs. The only thing is I have the hardware listed already and it's just sitting idle so I thought I'd make the most of it with some RAID combo. We were planning on putting server 08 r2 on there but from what I have seen that does not provide the benefits that zfs does in terms of data security.

I know that the lsi controller will check the parity data but that still falls short of the zfs data checksum. Is there anyway getting that data security without zfs?

 

[Ayah]

no, zfs does it object-wise while rocs do it block-wise. If you need to use the ROC, I'd do RAID60 with the 24 drives and set a backup somewhere else. A failure in the system would then only cost you your primary and not your secondary.

 

[AndyE]

Built a very similar system a while ago as an home server

Same MB
E3-1245v2 CPU
same memory
same drives 24*3TB WD Red drives
Raid controller is the new Adaptec 7 Series with Flash memory based backup system
Energy consumption is ca. 150 watt on the wall outlet when all disk spindles are on (idle).

I am using NIC teaming with 4x1GBit/s lanes, which is speed wise fully supported by the raid system. Both, read and writes.

I am also using disk deduplication which saves a lot of data - especially with RAW files, where during editing, only metadata is changed, but not the sensordata. Without deduplication, all copies in the workflow took up space on disk, with file-cluster-based deduplication - I don't mind anymore.

According to the log, uptime was 99,993% (Damned, I did not hit the magic 5-9's ... )

rgds,
Andy