Question Need advice on hardware/software for custom NAS

[djlenoir]

I am looking to build a custom NAS solution primarily to host my PLEX library. I have about 35TB spread across six WD 8TB Red drives using StableBit DrivePool. It works, but if I lose a drive I lose a lot of data. I am also down to about 10TB of free space, so I would like to start planning the next solution. I was thinking of getting a Synology system, but I feel like I can get more for my money doing it myself.

I do not have a server rack, so I would prefer a tower chassis if a decent one exists that I can stash away in the closet. If there are no good ones, then I can look into a small server cage to rack the chassis. Thinking minimum of 12 bays, but more is better, for SATA drives. I want to create a RAID (not sure which type yet so open to recommendations here as well) and be able to add disks over time (as I can afford them). I figure I can start with four or five 10TB drives, and add more over time.

What OS and software should I run? I know Windows with DrivePool, but I feel like there are better solutions available. Just want something that is reliable and works as a NAS for Windows.

 

[Muadib]

Unraid or Freenas are your best bet OS wise. I've used Unraid for just about 8 years, and have loved most of it. I run Plex in a docker, and it works well. Unraid isn't free though, but it's well worth the asking price. I liked that you could put any size drive in your server, and you would still have one drive for parity. Unraid's ram requirements was much lower than Freenas 8 years ago. I don't know if that's still the case though.

I'm in the process of moving over to a Qnap NAS, because I can't expand my Unraid server anymore unless I replace the HDD drives one at a time with larger drives. I've been doing just that, but the old girl just can't handle transcoding 4K videos well, and I have no room on the motherboard to add a graphics card to help.

Before I decided to go with a Qnap NAS, I searched google looking for a case that could handle 15 or more HDDs, but the only one I liked was rack mounted. Good luck with your search! Be sure & post back with your case choice.

Unleash Your Hardware

Unraid is an operating system that brings enterprise-class features for personal and small business applications. Configure your computer systems to maximize performance and capacity using any combination of OS, storage devices, and hardware.

unraid.net

TrueNAS - Welcome to the Open Source Storage Era

TrueNAS is the World's #1 Open Source Storage. Based on OpenZFS, it is trusted by millions and deployed worldwide.

www.freenas.org

 

[djlenoir]

Thank you @Muadib for this information. What made you decide to go with QNAP over a new custom Unraid server?

I looked at QNAP and Synology NAS solutions, but without expansion units the most I could see in in a non-rack mount case was around 12 bays (except one QNAP that had 12 3.5" plus 4 2.5" bays). It made me curious because you mentioned you were looking at a case with 15 or more drive slots and not rack mounted.

And if I may inquire further, what QNAP solution did you go with? I assume you are planning for expansion units down the road? Why QNAP over Synology (if you even considered Synology)? I appreciate your responses, because it may make me shift directions as well and just go with a complete solution like Synology or QNAP.

 

[aigomorla]

Plex though a NAS has issues.
Couple things you should note... do you need 4k transcoding?
If you do, then the best way is to do it though a windows machine with a NVEC capable videocard, otherwise you will need a fast CPU with about ideally 6 physical cores, and the jail/docker should be on a Solid State Drive.

For example, i retired out Plex from my FreeNAS when my 4k library started getting big, because the machine can not keep up with transcoding 4k content /w smi subtitles. You really need a strong transcoding hardware for this like Quicksync or NVEC.
Unfortinuately also HW Transcoding is not supported by FreeNAS, i am unsure about it for Unraid tho, as Unraid is Linux based, while FreeNAS is FreeBSD.
Another Note tho, i personally feel ZFS is far superior then Linux based File system.
ZFS is a feature of FreeBSD, this is why i am still on FreeNAS. However it is not as flexable as Linux.
Linux has a lot better compatibility with hardware over FreeBSD, but i prefer having the ZFS file system over HW Compatibility.

There is no out of the BOX NAS which can handle Plex + 4K unless you get to the real enterprise stuff, so if Plex is important, don't even bother with Qnap or Synology.

I use FreeNAS... but FreeNAS is a resource WHORE. It ideally likes to have 8GB of ECC REG Ram as base, and then a additional 1.5GB / TB of data as cache.

If your not too sensitive to noise i would suggest getting a repurposed / retired server off EBAY..
ZFS is also about the best type of file system out there, which FreeNAS. I would also setup some kind of redundant disk setup, like RAID-Z# where the # denotes how many disks you are allowed to lose before the end of the world happens to you with data.

FreeNAS community is also very TOXIC, so its a very steep learning curve if you want total customization, but for basic stuff, its pretty easy to setup and forget.

 

[djlenoir]

Thank you @aigomorla for the information. I failed to provide enough information in my original post apparently, so I apologize for any confusion I have caused. I really only intend the NAS for storage only. I do not plan to run my Plex server off of the NAS. I already have a Plex server running a i7 7700K and Nvidia card for NVEC support. I want to continue to use this as the server, but I need to expand the storage and figured that a network based storage solution would be the easiest.

You mentioned an old retired server. What retired server(s) would you recommend as a solid option for a Plex server if I was going to replace my current one and keep the storage local? I originally thought about getting a retired server, but I mostly saw rack mounted servers. I do not have a rack. I am not opposed to getting one, and maybe I should to organize the networking gear in the closet, but I have not looked into that option.

 

[aigomorla]

i would recommend you move the storage to another PC, keep the system as is for plex.

The way your describing to me seems you have no redundant fault protection for your data (you mention if you lost 1 disk, you would lose significant data), which is a big flag, especially when dealing with data capacity that large.

IF you have 6 disks, i would recommend you going at least a Raid-Z1 on FreeNAS if you go that route, which would mean you have a total storage capacity of 5 x 8 ~ 38TB (approx) of space with a fault tolerance of 1 disk. Meaning your array can handle losing 1 disk before all your data crashes.

The hard part would be migrating that data over to a temp storage while u reinitialize all the disks.

You can do a search on ebay for Supermicro FreeNAS servers.
I would recommend against getting a AMD one, as these are not Ryzen, but much older AMD Opterons.
They work, but they also eat a lot of power.

I would recommend you at least getting a Sandy Bridge which you can easily denote as the CPU's will be E5-26XX or e5-26xxV2's.
The third gen's is where things will get a bit more pricey E5-26xx v3's as those will require you getting DDR4 ECC Reg ram.

But these are full rated enterprise servers, and can easily handle NAS duty for probably as long as you need them to.
However note again, these are enterprise servers and they can get significantly LOUD with server fans.

If you need something quiet, you may need to either heavily mod the Chasis, or build one from scratch.

You do not need to put a rack mounted server inside a rack.
You can put them on top of coffee tables with UPS's on the bottom, or even lay them down on 2x4 wood on each side lifting them a bit off the floor.
My first rack server infact was plop'd ontop of a ikea coffee table, which i later transitioned to a rack, because now i have 3 rack servers.

 

[Muadib]

Thank you @Muadib for this information. What made you decide to go with QNAP over a new custom Unraid server?

I looked at QNAP and Synology NAS solutions, but without expansion units the most I could see in in a non-rack mount case was around 12 bays (except one QNAP that had 12 3.5" plus 4 2.5" bays). It made me curious because you mentioned you were looking at a case with 15 or more drive slots and not rack mounted.

And if I may inquire further, what QNAP solution did you go with? I assume you are planning for expansion units down the road? Why QNAP over Synology (if you even considered Synology)? I appreciate your responses, because it may make me shift directions as well and just go with a complete solution like Synology or QNAP.

The main reason that I went with QNAP, is the fact that I couldn't find a case big enough without it being rack mounted. Since you said that you would go to a rack, then you should take a look at Supermicro. I forget now, but they have a server that can take 22-24 HDDs.

As for my Qnap, I have the TVS-872XT. It's an 8 bay unit, but I can get the 12 drive expansion unit when I need it. I've only had time to watch one 4K movie on it though, and that was Joker. It had no issues transcoding it to 1080p, but I need to test it more. aigomorla is right, there are plenty of rack mounted servers on ebay, but I didn't want to go used. I also didn't try using a rack server outside a rack, because I already knew the wife wouldn't go for it. A buddy of mind has his in the bottom of a closet. However he's divorced now, and I don't know if that was a factor.

 

[mxnerd]

DIY IKEA server rack

IKEA Lack Table

Products

Browse our full range of products from dressing tables to complete modern kitchens. Click here to find the right IKEA product for you. Browse online and in-store today!

www.ikea.com

16 drive case + AMD X470 motherboard.

Are you a human?

www.newegg.com

16 drive case only

RPC-3216 3U Server Case w/ 16 Hot-Swappable SATA/SAS Drive Bay, MiniSAS connector

storage server rack rackmount case

www.ipcdirect.net

I also recommend UnRAID.

 

[Fallen Kell]

Check on eBay for used/refurb/off lease Supermicro 847 systems. You can usually find a barebones setup in the $500-800 range depending on CPUs and memory. The reason for the 847 is that you can easily replace the internal fan wall with 120mm quiet fans especially if you replace passive coolers on the CPUs with ones that have fans. Do a little research on servethehome as they had some good articles about the case, the SATA/SAS backplane, and the various configurations. There are definitely more desirerable versions depending on what you want to do, but from the sounds of it you can get away with any that have the SAS2 backplane (you only need the SAS3 if you were planing on SSDs in the RAID array). The system will fit 24x3.5 hot swappable SATA/SAS disks as well as has a bracket for 2x2.5 inch disks internal.

I upgraded mine with 40GB networking and replaced the heat sinks and fans and it is extremely quiet for a server (quieter than many desktops that I have had in the past). And considering it can use ATX form factor motherboards, you can easily replace the guts if you want in the future. You can also find the bare case for around $200, which is a good deal as well, but you will need to verify what backplane it has so you can get an appropriate SAS controller card with correct cables to connect. You also may lose out on some of the features such as indicator lights if you don’t have a supermicro motherboard, but not a huge loss.

 

[Muadib]

Check on eBay for used/refurb/off lease Supermicro 847 systems. You can usually find a barebones setup in the $500-800 range depending on CPUs and memory. The reason for the 847 is that you can easily replace the internal fan wall with 120mm quiet fans especially if you replace passive coolers on the CPUs with ones that have fans. Do a little research on servethehome as they had some good articles about the case, the SATA/SAS backplane, and the various configurations. There are definitely more desirerable versions depending on what you want to do, but from the sounds of it you can get away with any that have the SAS2 backplane (you only need the SAS3 if you were planing on SSDs in the RAID array). The system will fit 24x3.5 hot swappable SATA/SAS disks as well as has a bracket for 2x2.5 inch disks internal.

I upgraded mine with 40GB networking and replaced the heat sinks and fans and it is extremely quiet for a server (quieter than many desktops that I have had in the past). And considering it can use ATX form factor motherboards, you can easily replace the guts if you want in the future. You can also find the bare case for around $200, which is a good deal as well, but you will need to verify what backplane it has so you can get an appropriate SAS controller card with correct cables to connect. You also may lose out on some of the features such as indicator lights if you don’t have a supermicro motherboard, but not a huge loss.

Wow, you have a 40Gb network? How much are the network cards and switch? I just went to 10Gb myself. The switch was $500, cards were about $100.

 

[mxnerd]

Wow, you have a 40Gb network? How much are the network cards and switch? I just went to 10Gb myself. The switch was $500, cards were about $100.

NICs only, no switch.

 

[aigomorla]

Wow, you have a 40Gb network? How much are the network cards and switch? I just went to 10Gb myself. The switch was $500, cards were about $100.

yikes i think you paid to much.

mine were used but i got my cards for 40-50 each, and my silent new switch for 350.

 

[VirtualLarry]

my silent switch for 350.

LOL. On warm days, my D-Link 8-port 2.5GbE-T + 2x 10GbE SFP+ (one occupied) with dual 40mm fans, starts to get really loud. Should probably check my router, to see if someone is breaking in, but my NAS units are connected to my 25/3 connection, so at 3Mbit/sec upload, if someone were to break in, they wouldn't get many files very fast.

 

[aigomorla]

My 10gbe switch is passive, and its fast as hell.
Here i am doing a NAS -> Ramdisk transfer over 10gbe.

Of course i wont get that from Disk to NAS... i normally top off at around 600mb/s when transfering a 60gb bluray remux rip for example.

I didnt go the 40gbe route, because i wanted sfp+ and i wanted a real switch.
You can also always get transceivers for sfp to rj-45, but for 40gbe your pretty much stuck with infiniband.

 

[KentState]

You can actually find really good deals on the older Supermicro hardware. I originally purchased the 846 and have FreeNas running on the bare metal. 24 x 6TB drives in RaidZ2 (4*6 VDev) with 256GB RAM. It also hosts Plex since I'm not concerned with transcoding. I just play 4k in it's raw format from it which can feed more devices than people in the house.

A guy locally sold me the 826 and jbod encolsure for < $400 which was a steal. It runs Windows 2019 with Hyper-v to host a bunch of VM's. I ended up putting a pair of NVMe drives in it for the VMs, pair of SSD drives to run the OS and then 8 * 10TB shucked WD drives. The JBOD enclosure was hooked up to the 846 chassis and runs a mirrored pool of SSD drives along with a couple 6TB drives as spares if something fails in the other pool.

I also picked up a bunch of Chelsio 10Gbe and 40Gbe cards. The servers are directly attached with the 40Gbe connection as well as the Windows HTPC that sits in the rack. They are all then connected to a Netgear 10Gbe switch that connects to the rest of the house. I've looked at 40Gbe switches, but it would require pulling fiber to other rooms in the house. Not a huge deal, but not something I want to mess with at this time.

 

[Muadib]

yikes i think you paid to much.

mine were used but i got my cards for 40-50 each, and my silent new switch for 350.

My cards were new, but you might be right on the switch. I got it from Amazon, but here it is from a 3rd party:

Amazon.com: Ubiquiti US-16-XG UniFi Enterprise 16-Port Managed PoE+ Gigabit Switch with: Industrial & Scientific

Buy Ubiquiti US-16-XG UniFi Enterprise 16-Port Managed PoE+ Gigabit Switch with: Switches - Amazon.com ✓ FREE DELIVERY possible on eligible purchases

www.amazon.com

 

[delonm]

I think the best bang for the buck right now in 10Gb switches is MikroTik. Most of their models support running either as a full managed switch or as a router. I picked these up for my 10Gb storage network using a mix of DAC's and fiber. The 24 port unit is a switch only, but they also have a PoE+ switch/router that competes with the Ubiquiti switch Maudib linked above. Of course, Ubiquiti's management software is hard to beat. The interesting thing about the 4 port MikroTik (besides the ability to turn it into a router) is the ability to power it via PoE or external power bricks (it has redundant power brick inputs for a total of 3 power options). It does get a bit warm however.

https://www.amazon.com/MikroTik-CRS305-1G-4S-Gigabit-Ethernet-RouterOS/dp/B07LFKGP1L/ref=sr_1_2?dchild=1&keywords=mikrotik&qid=1593009972&sr=8-2

https://www.amazon.com/gp/product/B0723DT6MN/ref=ppx_yo_dt_b_asin_title_o02_s00?ie=UTF8&psc=1

I have a mix of Chelsio and Mellanox NIC's using 10Gtek SFP's and DAC's and everything works great. I paid ~$40 for the Mellanox v3 NICs and ~$100 for the Chelsio T520 off of eBay. FreeNAS can be picky about 10Gb NICs so I spent the extra money on one that is proven to be supported.

I am using a 2U 12 bay X9 based Supermicro server for FreeNAS and a 1U X9 SM server for Proxmox, both off eBay. Last night I setup two iSCSI drives on my Windows PC to use for Steam and GoG games storage. Still paying around with the best way to setup storage connectivity for Proxmox.

I have to say, playing around with a home lab has been very educational, if not a bit frustrating at times.

Now I am trying to find a decent rack mount UPS.

David

 

[Muadib]

View attachment 24000
[/QUOTE]

Now I am trying to find a decent rack mount UPS.

David

I always liked tripp Lite. They make quite a few models:

USV-Notstromversorgung durch Akkus | Eaton

Tripp Lite offers a wide selection of UPS systems, providing battery backup and power conditioning for servers, networks, desktop PCs and sensitive electronics.

www.tripplite.com

 

[delonm]

Thanks Muadib. Tripp-Lite is definitely on my list. I plan on culling the list down to 3 or so and then seeing which of those is on the supported list for NUT so I can have my FreeNAS server power down automatically in the event of a power failure. I think something in the 1500VA/900-1000W range should be adequate for my rack with only two servers and a few switches.

I was a bit shocked to see how much more expense the rack mount form factor is compared to the freestanding towers for the same specs.

 

[KentState]

I actually went with towers and just have them on the bottom of my rack. The smaller 300w at the top is for my networking equipment and was rather pricey.

 

[aigomorla]

I think the best bang for the buck right now in 10Gb switches is MikroTik.

spend a bit more and get the 309.

Amazon.com: MikroTik 9-Port Desktop Switch, 1 Gigabit Ethernet Port, 8 SFP+ 10Gbps Ports (CRS309-1G-8S+IN): Computers & Accessories

Buy MikroTik 9-Port Desktop Switch, 1 Gigabit Ethernet Port, 8 SFP+ 10Gbps Ports (CRS309-1G-8S+IN): Switches - Amazon.com ✓ FREE DELIVERY possible on eligible purchases

www.amazon.com

Its is totally worth it over the 305 as the 309 is fully passive, does not heat up thanks to the gignormous heat sink with heat pipes.
It also has 8 SFP+ ports.

I use the 309, and absolutely love it on SwOS.

 

[Fallen Kell]

Wow, you have a 40Gb network? How much are the network cards and switch? I just went to 10Gb myself. The switch was $500, cards were about $100.

The card cost about $30, and the switch I picked up for $200. Many enterprises are upgrading to the new 25/100Gb standard and as such are ditching their 10/40 gear.

You have to look around, I got some decent deals, but they are out there.

 

[Fallen Kell]

I didnt go the 40gbe route, because i wanted sfp+ and i wanted a real switch.
You can also always get transceivers for sfp to rj-45, but for 40gbe your pretty much stuck with infiniband.

You can find “real switches” with 40Gb. Mine is a layer 3, with 4xQSFP+, 8xSFP+, and 24xgbe ports. It is standard networking, not infiband or some other specialized format.

That said, it can be a pain to setup properly and you need to really think what you are doing. It isn’t a consumer or pro-sumer level switch but enterprise class.

The funny thing is I paid more for the network cable than for the network card. The card is actually a 2 port 40g card, but the internal connection over PCIe limits the card to only full bandwidth on one port, with the second port designed for simple redundancy. I also had to flash the card to change its mode (it is capable of being configured as infiniband or standard networking, you just need to flash the appropriate firmware from the manufacturer which can be a pain).

 

[aigomorla]

You can find “real switches” with 40Gb.

Im sorry i should of said one real as in ones you could actually afford and that don't scream louder then a entitled 4yr old..
yes i know its full blown enterprise gear.

Normally people who go 40GBe go direct PC to PC, and not though a switch like you state, this is why i said real.

But Kell your statements are absolutely all correct.

 

[Muadib]

The card cost about $30, and the switch I picked up for $200. Many enterprises are upgrading to the new 25/100Gb standard and as such are ditching their 10/40 gear.

You have to look around, I got some decent deals, but they are out there.

Someone in another thread a few weeks back said pretty much the same thing. It was too late for me though.