Building a PC-based NAS

[aigomorla]

+1 about not needing a 650W PSU

u can also get away with a corsair CX430 which usually go for 40 dollars shipped.

 

[Balforth]

Great info... I definitely would like to be at a cheaper price point. I only have 4 2TB drives, but obviously I was trying to give myself some room for expansion. However, in the 4 years I've been running a NAS, I've never broke ~2.5TB in stored data, so maybe to save some cash I'll just stick with 4 SATA ports. I'll try to come up with a new celeron build with a smaller psu. But I'm still confused about how to choose the right motherboard w/ECC RAM support

 

[code65536]

ECC RAM is nice to have, but not mandatory. It adds to the robustness of the system by protecting against corruptions that result from errors in memory (which are pretty rare). You're not going to see ECC RAM in the consumer market, but it is widely used in the enterprise market.

The guide in the FreeNAS forums skew very heavily towards people who are building for mission-critical systems; most of us who are doing this for our own personal home use don't need to go to such lengths. Keep in mind that a FreeNAS system on consumer-grade hardware without things like ECC RAM is still much better than, say, a Synology.

 

[aigomorla]

Great info... I definitely would like to be at a cheaper price point. I only have 4 2TB drives, but obviously I was trying to give myself some room for expansion. However, in the 4 years I've been running a NAS, I've never broke ~2.5TB in stored data, so maybe to save some cash I'll just stick with 4 SATA ports. I'll try to come up with a new celeron build with a smaller psu. But I'm still confused about how to choose the right motherboard w/ECC RAM support

u wont even require 8gb of ram for 2.5tb of data...

i heard a good rule of thumb is around 1gb per 1tb of storage u put in raid-z

if ur not even using raid-z, u wouldnt need more then 2gb even.


Also you wont require ECC...
its a fault safe device however for the everyday joe its overkill.
for a nas which is basically a media server / file server / network drive, ECC is a extra option / feature which isnt manditory.
And yes the FreeNAS unit will still be greater then a synology unit even without ECC.

 

[Balforth]

Ok, how about this build:

case/psu combo - Rosewill Ultra High Gloss Finished MicroATX Computer Case with 400W ATX 2.2 12V Power Supply, Black R363-M-BK - 39.99
cpu - Intel Pentium G3220 Haswell 3.0GHz LGA 1150 54W Dual-Core Desktop Processor Intel HD Graphics BX80646G3220 - 69.99
mobo - ECS H87H3-M(1.0) LGA 1150 Intel H87 HDMI SATA 6Gb/s USB 3.0 Micro ATX Intel Motherboard - 78.99
memory - Crucial Ballistix 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800) Desktop Memory Model BLE2KIT4G3D1608DE1TX0 - 69.99

Not familiar with the motherboard manufacturer, but the specs seem pretty good -- 32gb of ram and 6 sata ports, and it has nothing but 5 stars. It does have a Realtek network chipset, so I don't know if I should buy an Intel PCI nic. Also, the case has a 400w power supply, but it's so damn cheap. It only has 5 drive bays, but I guess if it came down to it and I needed to add that 6th drive I could figure it and rig something up. Also not entirely sure about the RAM, I just picked something, but I guess I'd have to look at recommendations from the motherboard manufacturer.

So bottom total price now comes to $258.96. Quite a huge savings from my first build. Assuming I might be able to sell my ReadyNAS for around $200, I'm looking at a total cost of $60.

Or save the ReadyNAS chasis and upgrade the my drives in the custom built NAS at somepoint, then re-use my WD Red drives in the ReadyNAS as a full NAS back-up.... hmm...

 

[mfenn]

Well everybody seems to say you can run FreeNAS on old crappy hardware and it rocks, but there's a sticky in the hardware forum over on the FreeNAS page about buying hardware, and the recommendation there is to not use any consumer-based hardware, only professional grade servers with xeon processors.

http://forums.freenas.org/threads/so-you-want-some-hardware-suggestions.12276/

That's a fine guide for somebody building a server with a budget of $1500 before drives, but we live in a world of limited resources and compromises. Aigo is right that FreeBSD 9 has much better compatibility with the typical NIC and SATA controllers found on consumer boards.

By the way, ZFS is no more or less vulnerable to memory bit flips than any of other filesystem that caches data in memory (read: all of them). It's always nice to have ECC on any server, but adding ZFS does not suddenly make ECC a requirement.

Also, I'm not even sure how to verify memory is ECC besides counting the chips, but most memory has heat sinks across them now so you can't. A lot of the stuff I looked at on newegg didn't say whether or not it was ECC/parity.

If the memory doesn't say ECC, then it's not ECC.

 

[mfenn]

Ok, how about this build:

case/psu combo - Rosewill Ultra High Gloss Finished MicroATX Computer Case with 400W ATX 2.2 12V Power Supply, Black R363-M-BK - 39.99
cpu - Intel Pentium G3220 Haswell 3.0GHz LGA 1150 54W Dual-Core Desktop Processor Intel HD Graphics BX80646G3220 - 69.99
mobo - ECS H87H3-M(1.0) LGA 1150 Intel H87 HDMI SATA 6Gb/s USB 3.0 Micro ATX Intel Motherboard - 78.99
memory - Crucial Ballistix 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800) Desktop Memory Model BLE2KIT4G3D1608DE1TX0 - 69.99

Not familiar with the motherboard manufacturer, but the specs seem pretty good -- 32gb of ram and 6 sata ports, and it has nothing but 5 stars. It does have a Realtek network chipset, so I don't know if I should buy an Intel PCI nic. Also, the case has a 400w power supply, but it's so damn cheap. It only has 5 drive bays, but I guess if it came down to it and I needed to add that 6th drive I could figure it and rig something up. Also not entirely sure about the RAM, I just picked something, but I guess I'd have to look at recommendations from the motherboard manufacturer.

So bottom total price now comes to $258.96. Quite a huge savings from my first build. Assuming I might be able to sell my ReadyNAS for around $200, I'm looking at a total cost of $60.

Or save the ReadyNAS chasis and upgrade the my drives in the custom built NAS at somepoint, then re-use my WD Red drives in the ReadyNAS as a full NAS back-up.... hmm...

Just say no to PSUs that come with $40 cases. The case itself costs around $20-25, leaving $10 for the PSU plus a couple bucks for profit. A $10 PSU will blow up your system.

Also just say no to ECS motherboards, they are absolute bottom of the barrel trash.

Luckily, you can fix both of these issues without spending a ton of money.

ASRock H87M $80
Corsair CX430M $40 AR - modular not required, but it is cheaper than the normal CX430 right now
Some Cheap Case $24

 

[Balforth]

Great advice... here's iteration 3:

case - DIYPC Skyline-A Black Steel USB 3.0 ATX and Micro-ATX Mid Tower Computer Case with 1x 80mm Fan - 18.99
cpu - Intel Pentium G3220 Haswell 3.0GHz LGA 1150 54W Dual-Core Desktop Processor Intel HD Graphics BX80646G3220 - 69.99
mobo - ASRock H87M LGA 1150 Intel H87 HDMI SATA 6Gb/s USB 3.0 Micro ATX Intel Motherboard - 79.99
memory - G.SKILL Sniper Low Voltage Series 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800) Desktop Memory Model F3-12800CL9D-8GBSR2 80.99
psu - CORSAIR CXM series CX430M 430W ATX12V v2.3 80 PLUS BRONZE Certified Modular Active - 49.99

Slightly higher price that the last build - $299.95, but still 50% the cost of the Synology I was looking at.... And the new case gives me 6 internal 3.5" bays for the full ability to add 2 more drives.

Is the Qualcomm onboard ethernet OK? Any other considerations I'm overlooking?

 

[mfenn]

That looks OK, but I'm not sure why you changed out the memory for a more expensive kit. Really, you just want the least expensive DDR3 1600 8GB 1.5V kit, which is this Team one at $54 right now.

As for the mobo, there is a driver in FreeBSD (alx(4)), but I couldn't find any definitive source stating that it was included in FreeNAS. The ASRock H87M Pro4 is an alternative with an Intel NIC.

 

[Balforth]

I switched up the memory based on the memory compatibility list for that motherboard. That team vulcan ram doesn't seem to be on the list. I've been looking at the DDR3 1600 4gb choices listed here, but they all seem to be a bit more pricey.

This is about the cheapest I can find on the list:

AVEXIR Standard Series 8GB SDRAM (4GB X2) Dual Channel CL9 240-pin DDR3 1600 (PC3 12800) Desktop Memory Module Model AVD3U16000904G-2SW

Thanks for looking up the mobo in FreeBSD/FreeNAS... I'm assuming you're talking about the network chipset? I swapped out my mobo for the Pro4 you listed. Is that one in there?

 

[Balforth]

Well, one of the mods over on the freenas forum didn't like my build at all. I think he is a big ECC RAM advocate. I put together another build using ECC memory...

ram - Kingston Technology ValueRAM 8 GB 1333MHz DDR3 PC3-10666 ECC CL9 DIMM Hynix M Server Memory (KVR13E9/8HM) 112.99
mobo - SUPERMICRO MBD-X9SCL-F-O LGA 1155 Intel C202 Micro ATX Intel Xeon E3 Server Motherboard 152.99
cpu - Intel Pentium 2 LGA 1155 Processor BX80637G2030 63.49
psu - CORSAIR CXM series CX430M 430W ATX12V v2.3 80 PLUS BRONZE Certified Modular Active 49.99

I'll have to recycle an old case for now. This brings the total to $379.46

 

[aigomorla]

ROFL...

OK.... ECC... if ur gonna map the network drive, and then use it as a storage dump for high I/O work where every last info needs to be hashed / checked / double checked... fine.. install ECC..

Otherwise, leave ECC for a full bloated server which does exactly that...

Balforth i know MANY MANY who have run freenas without ECC because they couldnt afford a full system with ECC.

ECC usued to cost 4x the price on normal ram... its not anymore due to all the data centers which flood retires them.

The board and cpu requirement for ECC is also MEH... unless ur looking at a full enterprise multi socketed board.
Id rather go gamer style then over a single socket enterprise board, because gamer boards have more quality then single socket enterprise boards.
And if u ask anyone will they run a multi processor system on freenas.... they will laugh at you unless u say its one of the Virtualized clients on that server.


I am fully aware your trying to save budget... id suggest you go back to Mfenn's build and use the excessive money on DRIVES instead of RAM.
If i had to pick between a ECC NAS vs a Larger CAP nas...

NO BRAINER... LARGER CAP!!! Unless I was writing highly data sensitive material to that NAS.

MOVIES / Music / Media files / Pictures are not highly data senstive... if they were, people on desktops would be losing data all the time.
When was the last time u had a write data error on your desktop PC where it didnt copy something correctly?

now if u had 50 people writing to that NAS and reading from it every second... well ECC may be a option which is more feasible for you.
However a NAS for a family of less then 5, where at most 2-3 people will be on it at the same.... ECC? WHY?

 

[Balforth]

LOL I really do believe you! It's just funny with the guys on that board so adamant about their stance and people here saying don't buy into it! They seem to preach that non-ecc RAM can completely trash your entire array.

Either way, I'm not going to be purchasing any new drives at the moment, just re-using my WD Red drives. So my only choice is between the two builds....

Let me ask you this... if you had the non-ecc gamer build and the server build sitting side-by-side on a table, all other things equal, and you had to pick one, which would you chose? Do you really think that SuperMicro motherboard is lower quality than the ASRock motherboard?

And one final question about ECC RAM. 99.9% of the writing that gets done on my NAS is copying tv shows that have been downloaded by my HTPC to my NAS. In the event of a bit flip on non-ECC RAM on data that is to be written, will it really screw up the whole array, or will it only corrupt the file that's currently being written?

I'm really sorry if I'm frustrating you! But I am really enjoying the conversation and learning a lot!

 

[Gillbot]

If you are set on ECC, I have a 4x4gb ECC set of ram here pulled from my server replaced with non-ecc I can sell you.

 

[Balforth]

I'm literally laughing out loud.... so is that one more vote for not needing ECC in my NAS?

I'm not set on anything right now, just have people very adamant on two separate sides of this argument and I'm trying to weight it out!!!

 

[Boondox]

I've never used ECC memory in my personal storage servers, and I don't see the need. I'd be more worried about hard drive failures. With motherboards, I'm currently using an ASRock Z77 Pro3, with the add-on of a server grade Broadcom PCI-E 1x NIC from an actual Dell PowerEdge server. It happily pushes over 120 MB/s read speed, and not a single hiccup since it was built.

 

[mfenn]

And one final question about ECC RAM. 99.9% of the writing that gets done on my NAS is copying tv shows that have been downloaded by my HTPC to my NAS. In the event of a bit flip on non-ECC RAM on data that is to be written, will it really screw up the whole array, or will it only corrupt the file that's currently being written?

Anything is theoretically possible, it's all a question of what's probable. The probability of single bit flip is very low (10^-14 per bit per hour). On an 8GB machine, you will likely see one bit flip every 1.4 years. The probability of the bit being flipped in actively used memory is lower than that. The probability of the bit flip making it to disk perhaps causing one pixel to be a different color in a video file is lower than that . The probability of the bit being flipped in all redundant copies of the array metadata is lower than that. So yeah, if you ran your array for a century, you might want to get ECC. Otherwise, I think you'll be fine.

I switched up the memory based on the memory compatibility list for that motherboard. That team vulcan ram doesn't seem to be on the list. I've been looking at the DDR3 1600 4gb choices listed here, but they all seem to be a bit more pricey.

The Memory QVL is just a list of memory that they've tested, not an exhaustive list of all memory that is compatible. I'd stick with the Team kit.

 

[Gillbot]

I might have him covered on ram.

 

[Balforth]

Thanks to all for the advice. Please don't be mad at me but given the sweet deal on 16gb of ECC RAM from Gillbot, I decided to go the ECC route. I ended up building it for the exact same price the non-ecc build would have cost with 16gb of new RAM. Here's the final product list:

ram - Ddr3-1333 4Gb/512Mx72 Ecc Cl9 Server Memory
ram - Kingston 4GB 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10600) ECC System Specific Memory Model KTH-PL313E/4G
mobo - SUPERMICRO MBD-X9SCL-F-O LGA 1155 Intel C202 Micro ATX Intel Xeon E3 Server Motherboard
cpu - Intel Pentium 2 LGA 1155 Processor BX80637G2030
psu - Seasonic 360W 80PLUS Gold ATX12V Power Supply SSR-360GP
case - NZXT Technologies Source 210 Computer Case (Black)

Now for the fun part... getting everything temporarily copied off the NAS, getting the FreeNAS machine built and figuring out how to configure it, and finally getting everything moved back.

I think I'm going to go with RAIDZ1. I can't wait to see how this effects performance when it's all done!

 

[mfenn]

Nothing wrong with getting ECC if the price is right. It's just not worth paying extra for.

 

[Balforth]

Just an update... I have my FreeNAS system up and running, and I don't think there's anything I can do to cause the buffering on XBMC I was seeing before.

I initially restored my TV shows to my media share (1.6TB). When that was finished, I started on my movies (1TB). While the movies were in the process of being written to freenas, I could a) watch TV shows on XBMC, b) process/move new files to the nas, and copy large files back and forth between the nas and htpc.

In the midst of all this traffic, copying files from the NAS averages between 70-80MB/s. I couldn't be happier!

I currently have it configured to allow everybody to read from the media share, but only the user that processes/moves files to the NAS with write access, I have a "public" share that anybody can read/write to, i have a private share, with a subdir for each member of my family. Each one of those folders is owned by the specific family members user/group, and permissions set to 770 so nobody else can browse. And finally, I just set up two AFP shares for my wife and daughters MacBook Pro's time machines to use for backups.

Have yet to schedule scrubs or SMART, and need to figure out what backup plugin to use and how I'm going to go about that... using internal SATA drives (still have 2 free SATA ports) vs using USB drives vs configuring my old ReadyNAS as a backup unit.

 

[mfenn]

Awesome! Glad you like it. Post up some zpool status porn when you can. :awe:

For ZFS scrubs, I think that weekly is more than sufficient. I'd also set up a daily snapshot job and retain those for a month or so. Since you have a primarily additive workflow, the snapshots won't cost you much in terms of storage space (effectively only changed/deleted blocks "count against" you).

 

[dave_the_nerd]

Post up some zpool status porn when you can. :awe:

Ummmm... like, seriously?

  pool: zfs_volume
 state: ONLINE
  scan: scrub in progress since Mon Dec  2 20:22:14 2013
        21.3G scanned out of 2.99T at 209M/s, 4h7m to go
        0 repaired, 0.70% done
config:

        NAME                                            STATE     READ WRITE CKSUM
        zfs_volume                                      ONLINE       0     0     0
          raidz1-0                                      ONLINE       0     0     0
            gptid/545e3d37-327b-11e3-92bd-300ed50f3e11  ONLINE       0     0     0
            gptid/54d7e178-327b-11e3-92bd-300ed50f3e11  ONLINE       0     0     0
            gptid/5551e59d-327b-11e3-92bd-300ed50f3e11  ONLINE       0     0     0

errors: No known data errors

I just started a scrub so it would be doing something. Certainly never struck me as particularly interesting...

 

[mfenn]

Ummmm... like, seriously?

Hey now, I didn't want you see your dirty ZFS setup. :awe:

But seriously, I am interested to see how he ended up setting up his zpool, and zpool status is the most succinct way to do that.

 

[Balforth]

Well, this doesn't seem very exciting... is this all you wanted to see?

  pool: Volume1                                                                                                                     
 state: ONLINE                                                                                                                      
  scan: none requested                                                                                                              
config:                                                                                                                             
                                                                                                                                    
        NAME                                            STATE     READ WRITE CKSUM                                                  
        Volume1                                         ONLINE       0     0     0                                                  
          raidz1-0                                      ONLINE       0     0     0                                                  
            gptid/32f0d8e7-5794-11e3-afb9-002590d43e13  ONLINE       0     0     0                                                  
            gptid/335373f7-5794-11e3-afb9-002590d43e13  ONLINE       0     0     0                                                  
            gptid/33b7bef9-5794-11e3-afb9-002590d43e13  ONLINE       0     0     0                                                  
            gptid/341d20eb-5794-11e3-afb9-002590d43e13  ONLINE       0     0     0                                                  
                                                                                                                                    
errors: No known data errors

I ended up setting up my SMART and scrubs according to this post:

http://forums.freenas.org/threads/need-help-setting-up-smart-zfs-scrubs.16182/

Do you do anything fancy with your snapshots -- like storing them externally or keeping them backed up?