Build or buy a NAS?

[Lil'John]

Title basically states it.

Basic goal is to serve up music and movies. Also a basic central file location. No "useful" work will be done there.

Lower power would be nice.

I don't have a problem building/setting up a computer. Building my own would allow me an easy method for customizing the size of the case for my small network closest. Depending upon motherboard, I could easily upgrade hard drive capacity.

Buying one would make it nice to be ready to go.

 

[CiPHER]

Depends on your needs and especially how much time and energy you want to invest.

For a real NAS where you want to provide good protection to your files, building yourself is the only way to go. All the read-made NAS systems like Synology and Qnap use older technology that cannot detect corruption, let alone correct it. If you build a solution yourself, you can utilise ZFS - the 'best' way to store your files in this era.

ARM chips in ready made NAS are a little bit more efficient than DIY builds, but it doesn't matter all that much if you choose the right components. A DIY build can actually be more power efficient than a read-made solution if you compare x86 versus x86.

But you say the data is not really important, so then a solution 'from the shelf' would be an option too. It will be more expensive and you get very budget hardware for it, but it does mean you save time and energy building a solution yourself.

I help out many people starting their ZFS adventure, generally if you fall into this category it can be very much fun and also a good learning experience to build your own NAS and equip it with ZFS. It is up to you what solution you prefer.

 

[Emulex]

Get a gen7 hp microserver! it is a great half-way between buying and building!

 

[CiPHER]

Indeed it is. If you want to build yourself but have an easy time, that could be it.

But to be honest, if you want to spend a little bit more time and money, there are better solutions especially if you want ZFS.

 

[nk215]

Many businesses use QNAP/Synology to store their files w/o issue. ZFS is great on paper but I am not sure how important it is in practice. ZFS needs significant amount of memory to work effectively so it won’t be a cheap built using parts laying around the house.

I do use ECC when possible (my QNAPs do not support ECC, my Synology does) to minimize corruptions.
DIY NAS vs COTS NAS comes down to the CPU power. If you won’t need 2-3 transcoding streams then COTS is the way to go. Mobile apps are a very strong advantages of the COTS NAS.

If you only needs around 4TB then you can’t really beat the 2-bay COTS NAS. 5+ Bays is where DIY NAS gains cost advantage. If you decide to buy, make sure the NAS can saturate 1Gbit network. Some of the sub $100 2-bay NAS are just too slow.

 

[CiPHER]

Many businesses use QNAP/Synology to store their files w/o issue. ZFS is great on paper but I am not sure how important it is in practice.

Why RAID5 stops working in 2009.

This is exactly one of the dataloss issues that ZFS will counter.

ZFS needs significant amount of memory to work effectively

32 megabytes is not all that significant. And in all configurations ZFS uses less memory than virtually all other filesystems, including NTFS, FAT32 and Ext4.

The difference is that ZFS can use memory very effectively because it is smarter than the VFS-layer of the operating system, because it distinguishes between recent data (MRU) and popular data (MFU) which normal VFS does not and will throw away valuable caches agnostically.

ZFS will work fine with less RAM, but it won't be able to utilise all the speed potential of your harddrives. So with few RAM, your array might do 300MB/s instead of 700MB/s. That is the effect of more RAM for ZFS. Generally not a problem because gigabit will cap the speed anyway. More RAM can cache metadata which is very effective for file searching etc.

so it won’t be a cheap built using parts laying around the house.

Most ZFS builds i help people with are quite cheap low-end builds of 200 - 300 for the system excluding harddrives. You can even do about 100-150 if your budget is stretched. This is cheaper than a solution from the shelf, which generally is twice as expensive while you get slower hardware and legacy software.

I do use ECC when possible (my QNAPs do not support ECC, my Synology does) to minimize corruptions.

It will only protect against memory corruption, which is rather rare. Without ECC, ZFS can detect all and can correct at least some of the corruption caused by RAM bitflips. If it couldn't fix it, at least you know which files are affected. Also note that - even with ECC - you are still not protected against 3 bitflips; only against 1 (correction) and 2 (detection).

 

[Lil'John]

If I build, it will be with all new parts/case. No parts laying around the house.

I've got plenty of time and while I won't say money is not an issue, I consider my computer build budget as part of my entertainment budget.

The computers that will consume content are a mix of Windows 7 and 8.1 only.

The thing that sort of appeals about the Synology setup is the "very thin" operating system. Basically dedicated to serving up files.

_CiPHER_, thank you for taking the time to describe ZFS. Did you have a recommendation for an "OS"?

 

[dave_the_nerd]

Meh. Your requirements are pretty basic. Buy a prebuilt Synology and call it a week.

 

[CiPHER]

There are various operating systems dedicated to serve as a NAS. OpenMediaVault and XPenology for example, but they don't support ZFS.

The ones that do support ZFS are: FreeNAS, NAS4Free and ZFSguru. There is also napp-it, but it uses a different platform (Solaris/SmartOS) that you need to install yourself.

I'm a developer for the ZFSguru project, so do not take anything i say for granted. Yet, i think most will agree on this: FreeNAS supports the most features and is the most suitable for business use. NAS4Free is basically in between FreeNAS and ZFSguru and has an excellent track record - NAS4Free is what FreeNAS used to be in the past before FreeNAS got bought and got new owners (iX Systems).

ZFSguru is the least mature of all projects and has the least number of features. But it might be the most user friendly and 'easiest' ZFS distribution suitable for novice users, and has very few restrictions. For example, FreeNAS doesn't allow you to use the system disk for anything else. So using an SSD to install FreeNAS to is kind of a waste.

In the end, it is a matter of preference. I highly recommend you use Virtualbox to try out a few of the projects i mentioned and settle on one that you like. In case you don't know: Virtualbox is virtualization software that acts as a virtual pc. So you can perform a test installation and see whether the interface is to your liking.

Also know that you can switch between ZFS platforms; as long as you do not use any ZFS feature-flags that are uncommon. Feature flags are the new version 5000 ZFS implementation where features can be enabled or left disabled to preserve compatibility with platforms that do not have this feature. This means if you do not create your ZFS pool with all bells and whistles, it can work on all platforms. You won't be missing much anyway; the cool features are supported by all platforms.

If you want to know more about ZFS, feel free to ask. But you should also compare it with Synology. If you just want something working quick and have no hassle; the Synology gives you what you asked for. It won't give you the best protection to your files, and it is kind of the easy-route. The ZFS route can be a learning experience and a lot of fun, provided you have one or two persons you can ask for help when needed. On the Dutch forums i regularly provide such help and have inspired many to take it upon themselves to jump the bandwagon known as ZFS!

 

[Cerb]

Why RAID5 stops working in 2009.

This is exactly one of the dataloss issues that ZFS will counter.

ZFS mitigates some failure modes of RAID 5 better than plain RAID 5, but drive failure and URE it does nothing for. The two failures it can help with are silent corruption during write or at rest, for which RAID 5 is stuck not knowing the good stripes from the bad stripes, and that the RAID isn't a single blob to pass/fail on rebuild. ZFS RAID allows recovery of good files that regular RAID 5 may not. If a drive fails, or the drive reports that it cannot read a sector, there's no difference from RAID 5.

The data loss issues from drive failures and UREs (the latter of which is, luckily, below spec in reality) are just as much of a problem with ZFS as non-ZFS. But, that's why ZFS supports additional parity devices for parity RAID. The advantages of ZFS, and its follow-ons, like ReFS and BTRFS, are based on errors occurring differently than thought of in years past (in particular, that positive reports from hardware about errors cannot be trusted to be all of the errors affecting your data, and that networking and storage devices lie).

 

[nk215]

I read Robin Harris “Why RAID 5 stops working in 2009” a few years back. His math is off at a few places and those errors add up. Either ways according to Harris both of my arrays should be gone by now

Array1: 6TB out of 8TB raid 5 (5 disks). I create check sums for the data and check the checksum every month or so. I’ve done this in the last 4+ years w/o any error. (I always do the checksum before a full array backup just to make sure I don’t backup bad data).

Array2: Synology NAS, switched out 5 2TB to 5 4TB drives. I switch the drive out one-by-one so basically; the array rebuilt itself 4 times already. No error. This NAS as 12 bays so in the future, it will be rebuilt quite a few more time when I add new HDDs, I don’t expect any issue. (At the end, I wiped the array clean and created a raid 6)

Furthermore, RAID cards have been happy to ignore URE in rebuild process for a long time now. Sector data is gone but it’s far from dooming the entire array.
ZFS has been around for 10 years now and it hasn’t caught on. Maybe because the issue it addresses is not that big of an issue.

Having said all that, I would not recommend rebuilding an array (RAID 5 or 6), not because of a chance of losing data but because it’s just taking forever. Copy the data out/restore from a backup and rebuilt the array from scratch take much less time.

BTW, I consider 32G is a significant amount of memory. My main server has 24G and I do not hit that limit. One of my QNAPs has 1 gig and it never run out of memory doing file serving, rsyn, iSCSI duty.

 

[dave_the_nerd]

The ones that do support ZFS are: FreeNAS, NAS4Free and ZFSguru. There is also napp-it, but it uses a different platform (Solaris/SmartOS) that you need to install yourself.

It is also an available file system on FreeBSD and most Linux distros (with ZFS-On-Linux.)

If you just want to set up your own file server the old fashioned way without the pros and cons of a pre-canned "NAS OS".

Personally, I'd recommend Ubuntu. (Lots and lots and lots of tutorials.) I use FreeBSD, because I was importing zpools from a FreeNAS box and at the time, ZoL didn't have all the feature support implemented. (FreeNAS implements bleeding edge features because they're like that, I guess.

 

[CiPHER]

ZFS mitigates some failure modes of RAID 5 better than plain RAID 5, but drive failure and URE it does nothing for.

I agree it does nothing extra versus RAID5 to cope with disk failure.

But URE's? Of course ZFS protects against that. That is the whole point!

Even on a single disk without any redundancy (no mirror/RAID-Z) - ZFS partly protects against URE because of the redundant metadata. So any URE that affects a sector in use by metadata, will be instantly corrected by ZFS. The same applies to data stored with copies=2. So you do not need volume redundancy to protect against URE. But copies=2 on your data can be very 'expensive', and it does not protect against disk failure only against corruption/bad sectors (URE).

With redundancy, like RAID-Z, you are basically immune to URE. With one disk failed, you revert to the behavior described above.

RAID-Z2 can have one failed drive with full protection against URE, while it can have two failed drives with full protection against metadata corruption from URE - so ZFS itself will survive maybe some files don't.

A traditional RAID5 can fail with just 2 bad sectors. Or one drive missing + 1 bad sector. This is what Robin Harris' article is all about. That after a disk failure you begin rebuilding your array and because this takes a long time and stresses the disks, any remaining disk with a bad sector might cause the entire array to be considered failed by the RAID controller. Unless you have TLER disks, the disk with the bad sector will also be dropped. Now you are in the process of rebuilding a RAID5 but one of the member disks is missing. That means that without some expert help to get that one disk going again, you cannot access your data. While recovery is possible, many home users simply forfeit their data and re-create the RAID-array and the filesystem and start from scratch.

Backups... they hear. But backing up hundreds of gigabytes of not-so-important-data is not very feasible to home users. They just want their one storage vault to be reliable.

While ZFS' ability to automatically repair corruption (self-healing) is often praised, i find that simply the detection of corruption is invaluable. For many users, losing one or two files in the worst case that can realistically happen, is not that bad. If only they KNOW about which files. They can re-download them or just consider them lost. But not knowing what you lost is bad, and discovering that after all this while your wedding photo's are corrupted is even worse. Now you may have backed up those a million times, but silent corruption can affect the backups as well. So my point is that data integrity starts with error detection and is followed by error correction.

 

[Cerb]

Even on a single disk without any redundancy (no mirror/RAID-Z) - ZFS partly protects against URE because of the redundant metadata. So any URE that affects a sector in use by metadata, will be instantly corrected by ZFS. The same applies to data stored with copies=2. So you do not need volume redundancy to protect against URE. But copies=2 on your data can be very 'expensive', and it does not protect against disk failure only against corruption/bad sectors (URE).

With copies, you may as well have RAID (and you are now using non-standard configs), and with just metadata, not enough of importance is protected. Redundancy is needed to protect against UREs, and ZFS does not change that. It enables better protection, but you have to go out of your way to make that happen. FI, saving off files to another local directory will safeguard your data just as well.

A URE with plain RAID 5 and a URE with RAID-Z1 are functionally identical. A URE in your data is the same regardless of FS, too (how much space does metadata take up v. data? Hardly any! So, it's much more likely there). What ZFS specifically does protect against, that is not protected against in most systems (BTRFS and ReFS being notable exceptions), and which appear to be ever more important as time goes on, are not UREs, but successful reads of bad data, which need OS-level or higher awareness to handle.

Those same two bad sectors still cause lost data, but won't usually stop you in your tracks. In a Linux RAID 5, you can still get the whole rest of the array back, too. ZFS isolates the loss to a smaller space than just the stripe, and allows easier IDing of what data is lost, but the data is still lost. Some other RAID implementations may or may not stop cold with a failed stripe. Likewise, Linux' RAID handles non-TLER disks just fine, along with degraded arrays.

 

[Emulex]

LSI megaraid controllers with SAS drives support multi-drive read verification and partial disk recovery-rebuild mode to combat these problems.

The problem is a multi-bit error using standard sata drives will pass bad data right through a "old school" raid controller to the o/s.

But there is a price for having to run checksums on every read and having to read redundant data from extra platters every time!

 

[CiPHER]

Sorry for the long reply. Don't feel obliged to respond to anything. Just reply to the parts you'd like.

Edit: in retrospect, i better create a new thread instead of disturb this thread, since the issues go deeper than is relevant for this thread. I also don't know if you want to discuss this, Cerb? It is fine by me if you don't. I also wrote it thinking i could use it for my 'tweakblog'.

Anyway, new topic here: Bad sectors: ZFS versus legacy RAID

 

[Lil'John]

_CiPHER_ and Cerb, thank you for the good information and discussion.

I think I'll go forward with doing a home built NAS from scratch.

From my reading, it sounds like memory is the best thing for a NAS and not raw CPU power. So I'll assume that a E5-2695v3 would be a bit of overkill for this applicationD: I'm still trying to find an excuse to get one

CPU:
Would something like an i3-4370 be a good start for a cpu? (2 core, 4 thread, 54 watt) I get a steep discount on some Intel processors through work and this is the lowest one.

Motherboard:
Is it worth looking at a server class MB versus "normal" MB? C222, C224, or C226 chipset vs 87/97 chipset.

Will FreeNAS or similar take advantage of dual network ports?

Is it better to get more SATA ports on the board or use PCI addon cards? Basically, will a PCI card be faster than on MB SATA ports?

As an example(probably at high end of my MB budget):
ASRock E3C224D4I-14S
http://www.newegg.com/Product/Product.aspx?Item=N82E16813157486

Case:
Is there a good case to look into? I'm sort of space constrained with a "thickness" of 7" max in my storage cabinet.

Is a rackmount chassis the best bet? Or are there better "drive packed" cases to look at?

For example:
ARK 3U390A - 3U rackmount
http://www.newegg.com/Product/Product.aspx?Item=N82E16811128080

 

[Squeetard]

Build your own or buy a barebones NAs and some good reliable drives. Do not buy a pre-built NAS. They put bottom of the bin cheapo drives in them and they fail.

 

[exar333]

Build your own or buy a barebones NAs and some good reliable drives. Do not buy a pre-built NAS. They put bottom of the bin cheapo drives in them and they fail.

This.

I love to build my own rigs, but my QNAP NAS has been great. It is small, compact, power-efficient and site behind my monitor. QNAP/Synology just 'work' and come with a lot of easy to setup features.

 

[LurchFrinky]

CPU:
Would something like an i3-4370 be a good start for a cpu? (2 core, 4 thread, 54 watt) I get a steep discount on some Intel processors through work and this is the lowest one.

Motherboard:
Is it worth looking at a server class MB versus "normal" MB? C222, C224, or C226 chipset vs 87/97 chipset.

That cpu is overkill, but if it is the lowest one you can get, then great.
You didn't specify if you were intending to go with ecc memory or not. Using a C22X chipset is a requirement for implementing ecc memory on a 1150 socket system.

 

[rchunter]

Depends on if you need transcoding or not. Synologys are great but they aren't so great at transcoding. I have a DS1513+ and I love it. But I built a unraid/xeon rig running plex for my serious stuff.

 

[Ayah]

That cpu is overkill, but if it is the lowest one you can get, then great.
You didn't specify if you were intending to go with ecc memory or not. Using a C22X chipset is a requirement for implementing ecc memory on a 1150 socket system.

ditto. C226 is more workstation oriented though so you'd prolly end up C222 or C224 depending on cost (technically, some expandibility related performance but typically negligible)

 

[rchunter]

Server boards usually have ipmi and ecc memory. That's a plus. But you have to decide if you need those features. I got a SM x10sl7-f and I love it. IPMI is great.

 

[Lil'John]

That cpu is overkill, but if it is the lowest one you can get, then great.
You didn't specify if you were intending to go with ecc memory or not. Using a C22X chipset is a requirement for implementing ecc memory on a 1150 socket system.

I could go lower through a box store... but the price I get on the i3 would get me Celeron-ish ranged CPUs at a box store.

I'm not above going ECC memory. The reason I looked at server motherboards was feature sets such as dual+ ethernet ports, more SATA ports, SAS ports, etc.

I'm also not above going with a BGA CPU/MB combo.

The goal of this system is storing files and serving media content(music and movies). I'm not planning anything useful run on the machine. With that in mind, the features I see as being useful for such a system are:

• SATA ports - more the merrier.
• Good network features - 1Gb and more of them?? (house is wired for 1Gb)
• Lower power for 24/7 running

I have a 3930k system to rip/convert my media content. So I have plenty of horse power.

 

[dave_the_nerd]

CPU:
Would something like an i3-4370 be a good start for a cpu? (2 core, 4 thread, 54 watt) I get a steep discount on some Intel processors through work and this is the lowest one.

That would be fine. Even a Pentium/Celeron would probably do the job.

Motherboard:
Is it worth looking at a server class MB versus "normal" MB? C222, C224, or C226 chipset vs 87/97 chipset.

I think so. (See sig.)

Will FreeNAS or similar take advantage of dual network ports?

For the last version I used, not by default. It can be made to do so, but that becomes a effort/reward calculation - how much do you like tinkering with FreeBSD networking configs, how many clients do you have, will they ever be pulling/pushing enough data to swamp a single GbE, etc.

Is it better to get more SATA ports on the board or use PCI addon cards? Basically, will a PCI card be faster than on MB SATA ports?

Onboard SATA ports will tend to be faster, depending on the card. But in most cases, the spinning hard drives will be the limiting factor, and those are still way faster than the network itself. IMO it's not worth worrying about.

Case:
Is there a good case to look into? I'm sort of space constrained with a "thickness" of 7" max in my storage cabinet.

7" is narrow. But most racks are 19" wide. Do you mean you have 3U free?

Is a rackmount chassis the best bet? Or are there better "drive packed" cases to look at?

There are a number of ITX-sized 4-bay NAS cases in the $150-$200 range. I use a Fractal Design Node 804, which is fairly compact and has 8-12 HD bays (depending on who's counting and what size drives they're using), but they are not hot swappable.

A rackmount case will tend to be superior, but it will also tend to be more expensive. If you have an equipment rack in your home, I'd assume you already have appropriate power, cooling, noise abatement, etc.?