Best System for NAS with Software RAID

[citsacras]

The reason I post this topic in the *nix forum is because I am looking for recommendations specifically with regard to Linux software raid.

I was interested in getting a NAS solution, something like an Infrant ReadyNAS, but for the cost I can build my own low-power server with much greater flexibility.

The primary use of the server will be serving media. I am thinking a software RAID5 or RAID6 configuration on a UPS will be best for my storage needs. The network is runs through a 1000/Mbit switch.

It looks like the ICH10R southbridge offers the most SATA-II features, but I will not be using on-board raid so that my disks remain portable. Will I lose any performance or features if I go with any cheap chipset that supports enough SATA disks for my needs?

I was thinking of the E5200 CPU (2.5ghz, 45nm, 2MB cache). From what I understand, I could probably get by with an even slower CPU, but will it affect RAID performance? I would also like to use this server as a BT seedbox, iTunes server, and VM server as well. No encoding will be done on the server.

Finally, how much RAM should I use? Will the amount of RAM impact the performance of software RAID? Is there a way to allocate some system memory as cache for the array, like how some HW controllers allow you to install a 1GB DIMM? Can a large amount of RAM minimize disk thrashing for

Finally, I am interested in an efficient, quiet, system that can remain on 24/7 and not waste too much energy.

The parts I am planning to use are:
Seasonic 430W PSU (already own)
1 GB DDR667 (already own)
Lian Li PC60 Chassis (already own)
iStarUSA 4in3 hot-swap SATA-II backplane (about $100)
Asus P5Q-EM motherboard (about $135)
Intel E5200 CPU (about $80)
Western Digital 320GB HD (boot drive)
4x or 5x Samsung Spinpoint F1 750GB HD (about $100 each)

Please help me build the best server I can!

Thanks in Advance,
Jonesy

 

[child of wonder]

Right now the AMD BE-2400 2.3GHz 45w dual core CPU is the best bang for the buck out there. I just picked one up for $25 AC from Newegg.

Based on what you're going to use the server for, the BE-2400 would be more than enough for a software RAID array. My server is running this CPU and 4GB of RAM. I'm using it as a VMWare server running 2 VMs (one is Debian Linux running Postfix and Apache, the other is a Win2k3 torrent box). It's using 2x250GB SATAII drives in software RAID 1 to host the virtual machine files and the host OS (Debian) and 3x1TB drives in software RAID 5 hosting HD media files, game isos, MP3s, pictures, etc.

Load average hovers around 0.20 on average.

 

[QuixoticOne]

The reason I post this topic in the *nix forum is because I am looking for recommendations specifically with regard to Linux software raid.

I would also like to use this server as a BT seedbox, iTunes server, and WM server as well. No encoding will
be done on the server.

You lost me on the WM server and iTunes bits .... those work on a UNIX server? I guess if you mean you'll be serving FILES / sharing "drives" and folders to another OS that iTunes or WM uses to store files on, yeah, sure, you can share your iTunes / WM files that way.

Or if you run iTunes / WM in a VM or something and have them connect to a shared drive being shared by the UNIX OS.. sure.

Anyway the very best most secure RAID filesystem I'm aware of is ZFS, ESPECIALLY if you're just using an ICH10 type controller and a bunch of cheap disks, i.e. no $200-$500 battery backed NVRAM hardware based RAID card.

The only three OS choices you have to get ZFS are: OpenSolaris / Solaris, FreeBSD, Nexenta.
Of those, I certainly commend OpenSolaris' current SXCE/ON build as being the most feature packed and most reliable ZFS implementation. FreeBSD is a little behind in their ZFS implementation and as such I would somewhat hesitate to trust it for critical uses until they take it out of "beta" and add more official support for ZFS. Plain old Solaris is fine, but it has some rough spots for general purpose use if you're used to something like LINUX so I'd say OpenSolaris is better.

http://www.opensolaris.org/os/.../zfs/docs/zfs_last.pdf
http://en.wikipedia.org/wiki/Zfs

http://opensolaris.org/os/downloads/

here's the one you probably want -- B97 (build #97) is current; in a week they'll have B98 out.
Sometime in November there will be a major new release of OpenSolaris that will integrate stuff between SXCE and the main OpenSolaris project; that may be worth waiting for if you have time.
http://opensolaris.org/os/downloads/sol_ex_dvd_1/

You can use virtualbox under opensolaris to run a virtual machine of whatever... LINUX, Windows XP, etc. if that helps. Personally I'd keep my iTunes / WM server / whatever on a totally different PC as my NAS/fileserver though!

Just set up a RAIDZ of your 3 disc drives that will be used for storage, create a filesystem in the pool, turn on the NFS / SMB sharing feature on your filesystems, and away you go.

A little rough around the sysadmin edges compared to, say, SUSE / Fedora LINUX, but the pure data reliability should be WAY better than anything you're going to get from LINUX once you get the box installed / stable / running. ZFS is just that much better.

CPU: doesn't matter... anything faster than an X2-4200 or E2140 should do in a pinch... if you could get something a fair bit faster it'd probably help your I/O throughput on the gigabit network and for doing the RAID parity calculations during times of heavy use. If it is mostly for LAN oriented media like HD-video then I suppose I'd use something more like an E8400 or whatever if you really want to saturate a gigabit network and get like 100MB/s sustained read/write throughput under heavy load.

RAM: 4GB or 8GB... it is cheap enough and it helps a lot for disk cache RAM. I guess you could survive with 2GB but why... $40 after rebate gets you 2x2GB crucial DDR2 sticks from Frys today.

ICH10: should be fine. I've done it on ICH9R.. I assume ICH10R/ICH10 should be fine. You don't WANT to enable chipset / BIOS RAID with ZFS anyway.. just JBOD mode in AHCI or whatever is best.

 

[QuixoticOne]

There are also:

http://www.nexenta.org/os

http://freenas.org/

I'm not quite sure I'd trust my data to FreeNAS even though it is probably excellent given the versions of software / filesystems it uses (and probably better than many commercial NAS units!), I think there are better choices.

I don't see why I'd run Nexenta when I could run OpenSolaris for the time being. Maybe eventually when it is more mature....

 

[Fallen Kell]

Well, personally I would look at building a system that is Solaris X86 compatible and install Solaris 10 onto it so you can use ZFS. In terms of software RAID, there isn't a better free solution available. It was designed to be used with Zettabyte filesystems. Yes Zettabyte as in 1,099,511,627,776 GBs. In fact the limit for a storage pool in ZFS is 256 ZBs!!!

It also has real data integrity checks, not just XOR bits. Data in RAID arrays can become corrupted without it being detected. However ZFS uses a secondary hash of the actual file being copied so it knows when something corrupted the file itself and can repair it.

ZFS is really the be-all, end-all software RAID. It is so simple to add new disks to a disk pool, make a snapshot backup of the data (so you can see/use previous versions of files), migrate the filesystem to another system (as simple as a export command (zpool export diskpoolname), disconnecting the disks, attaching to another solaris box (or fresh installation), and run an import command (zpool import diskpoolname), and ALL your previous configurations are moved with the filesystems and diskpools). You have quotas so you can create multiple filesystems on a single diskpool and allow them to only grow as you see fit. I really can't say enough about this. You have to try it if you are looking at building a network fileserver. Layer in samba and all your windows boxes can use it however you see fit.

 

[citsacras]

I'm planning to stay with an Intel platform because of the ICH10R southbridge is the only one that I am sure supports AHCI and SATA hot-swap. So even though the AMD 45w CPU is inexpensive and efficient, I'm not sure there is a motherboard chipset that offers that. I'm not planning to use the "Intel Matrix" RAID, but it appears that they have the best chipset for SATA period.

Looks like at least a couple recommendations for ZFS. The biggest concern for me is that the secondary functionality of the server would be more limited than with linux because there isn't as much software available. Also, I haven't used Solaris much since Solaris 8 and never on x86 so I'm not sure how much of a learning curve would be there.

I'm definitely going to look deeper into ZFS though, thanks for the suggestions!

~Jonesy

 

[Fallen Kell]

Solaris isn't as limiting as you think. To be honest with you, just about every GNU x86 app works. Take a quick look at http://www.sunfreeware.com/indexintel10.html to see what is available pre-compiled and pre-packaged for you. As for other apps/software, most will compile since you can install gcc, gnu make, etc., and that is usually the main requirements for most gnu software, especially since it is the same x86 architecture, the only difference is the Sun kernel, but the gcc compiler is what handles that. The only few things that will not work are things that specifically require things from the kernel. But things like firefox/mozilla/thunderbird, apache, mysql, php, all work.

Heck, it uses the same gnome desktop, so look/feel wise it is almost identical.

 

[QuixoticOne]

OpenSolaris actually runs most of the 'linux' oriented userspace tools these days, GCC, GNOME, Firefox, Thunderbird, et. al. There are sites like the following that offer a bunch of packages that aren't on the Solaris DVD:
http://www.sunfreeware.com/

Also you can just fire up a VitrualBox VM and run a LINUX guest under Solaris if there is any convenience in that.

Actually with a VirtualBox or VMWARE VM you could run a OpenSolaris GUEST under a LINUX (I'd look at Fedora or CenTOS) host. With the free VMWARE ESXi you could run an OpenSolaris Guest as well as a LINUX Guest.

I wouldn't try to do anything very intense on video / graphics on a proprietary GPU on a file server box; the proprietary GPU drivers for LINUX / UNIX aren't unknown to be somewhat unstable and OS incompatible. I wouldn't be doing much with audio either for stability reasons. No major hardware reliant stuff like scanning/printing/USB devices/etc. Other that that sort of thing you can run most any secondary "server related" things on a fileserver pretty well either directly in OpenSolaris or in a VM guest or something with little difficulty.

 

[Fallen Kell]

Originally posted by: QuixoticOne
OpenSolaris actually runs most of the 'linux' oriented userspace tools these days, GCC, GNOME, Firefox, Thunderbird, et. al. There are sites like the following that offer a bunch of packages that aren't on the Solaris DVD:
http://www.sunfreeware.com/

Also you can just fire up a VitrualBox VM and run a LINUX guest under Solaris if there is any convenience in that.

Actually with a VirtualBox or VMWARE VM you could run a OpenSolaris GUEST under a LINUX (I'd look at Fedora or CenTOS) host. With the free VMWARE ESXi you could run an OpenSolaris Guest as well as a LINUX Guest.

I wouldn't try to do anything very intense on video / graphics on a proprietary GPU on a file server box; the proprietary GPU drivers for LINUX / UNIX aren't unknown to be somewhat unstable and OS incompatible. I wouldn't be doing much with audio either for stability reasons. No major hardware reliant stuff like scanning/printing/USB devices/etc. Other that that sort of thing you can run most any secondary "server related" things on a fileserver pretty well either directly in OpenSolaris or in a VM guest or something with little difficulty.

You don't even need to go so far as to use VMWare, just fire up a linux zone....

http://www.virtualization.info...is-containers-for.html

 

[Fallen Kell]

duplicate post

 

[QuixoticOne]

Here's another NAS/file server appliance distribution I ran across last night FYI;
personally I wouldn't use it in place of OpenSolaris.

I also wouldn't replace a full UNIX/LINUX server of any kind with this if I wanted the box to have multiple roles and full administration capability.

If I just wanted a storage appliance with minimal sysadmin needs, though, this or FreeNAS might be attractive:
http://www.linux.com/feature/146861
http://www.openfiler.com/products

 

[Fallen Kell]

QuixoticOne, I beat you to that sunfreeware

 

[QuixoticOne]

Good point.

Originally posted by: Fallen Kell

You don't even need to go so far as to use VMWare, just fire up a linux zone....

http://www.virtualization.info...is-containers-for.html

 

[QuixoticOne]

Yeah that was funny when I noticed we'd posted at basically the same time.
I guess somewhere someone at SUN is smiling that we're evangelizing all the ZFS / Solaris goodness.

Originally posted by: Fallen Kell
QuixoticOne, I beat you to that sunfreeware