Drive Pooling for Windows

[Cerb]

Adding parity drives increases the number of drive failures. I don't know exactly how, but I assume it's some type of data sums + parity calc = x type of formula.

It's simple: the parity is duplicated. An added parity drive will just have the same data written to it as the first one.

ATA and SCSI specs disallow returning known incorrect data, so the granularity of everything is generally 512B or 4096B (though there are 520B SCSI, and probably 4160B, by now). Not being able to make use of a bit-string of unknown correctness, fancy ECC schemes are of no use, without extremely large stripe sizes, and/or distributing the data and codes across many drives.

 

[Carson Dyle]

The fact that something like FlexRAID works at the file level and not the drive/sector level, has to make a difference, doesn't it? Data isn't striped across the drives.

 

[Cerb]

The fact that something like FlexRAID works at the file level and not the drive/sector level, has to make a difference, doesn't it? Data isn't striped across the drives.

Working at a file level allows you to keep parity over a JBOD, such that you can read all the data without keeping a coherent RAID array. It's not like a RAID array, in the sense that a RAID 0, 4, 5, 6, etc., has fixed-size fixed-location stripes, while a file-based system has to work on files in a file system, which vary in size and location.

If RAID is a good option for your uses, then everything offered by a system like FlexRAID is all downsides. If a server with a RAID array is a small hill of PITAs for you, OTOH...

It may not be striped across the drives, but to be able to use 1 drive for parity over say 3 drives of data, the parity has to be made by utilizing the data on all 3 drives. Being file-level, it will have to keep track of what file data it has used for parity so far, and be able to perform parity operations across different amounts of data, on different numbers of disks, making it much more complicated. But, the same principles will apply.

For example, let's say there's 211GB used on drive A, 440GB on drive B, and 320GB on drive C. Drive D, storing parity, will need 440GB of space to store all the parity.

211GB of it can be protected most efficiently, with drive A,B, and C data, so will get protection like a 4-drive RAID 5.

109GB of it will only be protected by B and C data, so will be protected like a 3-drive RAID 5.

The remaining 110GB of drive B will not get such protection, and will have to have as much space used to protect it as it takes up, to be able to guarantee recovery. However, as new data is added to the other drives, re-calculations can allow this to shrink, and the other parity sets grow, so the space needed won't need to exceed the most filled drive (at least not by much--I'm sure there's some overhead for metadata and whatnot).