YANB (Yet Another NAS Build) - this thread is not like the others, I hope!

Page 5 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

frowertr

Golden Member
Apr 17, 2010
1,371
41
91
Easier could be one thing, but what about relative safety? What happens in mdadm when it encounters a URE during resilver? And in other areas, ZFS goes a long way to ensure data integrity.

How is ZFS's implementation of RAIDZ2 (basically RAID 6) any better with handling UREs than mdadm with Ext3/4/XFS/etc... with RAID 6? Two URE encounters during a RAID 6 resilver and you're hosed no matter the base file system.

Truly, I'm not trying to persuade you away from FreeNAS. I just don't 'get' the use of ZFS for a home NAS.

/shrugs
 
Last edited:
Feb 25, 2011
16,823
1,493
126
Mostly just because.

And I'd like the consistency between my OS X CLI experience and FreeBSD. And I've never touched FreeBSD before so, it's new waters to test.

Oh my god, there is no consistency there, don't even. :thumbsdown:

And if I used Linux, it would certainly be ZFS on Linux. After doing my research, ZFS is definitely what I want to utilize. And I figure if I'm going that route, I might as well take advantage of FreeNAS. And I like what FreeNAS offers in terms of interface and platform.

What exactly? Quirky BSD and a WebGUI?

There's still the question of total platform power use, which I suspect will be a [relatively] significant difference when comparing a standard setup vs the integrated SoC approach.

Not really. A CPU and a chipset is a CPU and a chipset - the only trick with the SoC is that they're binned parts, and they're throttled. People throw around TDP numbers like they're reflective of power use, but most Intel CPUs come in well under their TDP when under load, and everything since Sandy Bridge idles in single digits. The difference IRL between the Xeon-D SoC and a socketed E3 are likely insignificant next to the power of the dark side... err.... I mean, the amount of power required to keep all your HDDs spinning.
 
Feb 25, 2011
16,823
1,493
126
ZFS isn't really designed for 'production use' on Linux. At least not that I'm aware of yet. Ext4 and XFS are what you'd want for Linux if you went that route. I use Ext4 for just about everything. I've never quite understood the drooling over ZFS to be honest.

The devs have considered it production ready for a while. It's just (well, 'just' minimizes it - it represents a fair amount of effort) a Linux port of the same OpenZFS code that got dumped by Solaris and incorporated into FreeBSD. Sometimes it's a feature or two behind, which happens when you have multiple dev teams but I'm not sure how that matters, since ZFS has been "production ready" for years.

I'm getting ready to convert my home NAS box from Windows with Unraid to Linux with Samba/mdadm. Should make a fun weekend project when I find the time.

To my knowledge there is no way to 'cap' the clock speed within ESXi for a VM.
Are resource pools available in FreESXi?
 

frowertr

Golden Member
Apr 17, 2010
1,371
41
91
Dave, what production Linux distro has ZFS incorporated directly into it?

I don't doubt it will be incorporated in due time but hadn't seen any of the major players yet to do so.
 
Feb 25, 2011
16,823
1,493
126
That is impressive. How many drives in the pool?
This contextless screenshot again? Yes, it's true - a sufficient number of HDDs in an array will give you monster sequential IO numbers. That's hardly unique to ZFS.

Hint: the first question to ask is "how much ram is in the server?"
 

frowertr

Golden Member
Apr 17, 2010
1,371
41
91
BTW, if it seems like we are all getting snippy I think it's because we are ready for you to make a decision and buy something.

Sometimes you gotta either crap or get off the pot!

 

destrekor

Lifer
Nov 18, 2005
28,799
359
126
Oh my god, there is no consistency there, don't even. :thumbsdown:
Really? Damn. Oh well, don't really care in the end. I figured there'd be some similarity between FreeBSD and OS X considering the Darwin kernel's origin. Then again, Darwin evolved from BSD so I don't know what differences were incorporated in FreeBSD in comparison to BSD. And of course they've developed it themselves so of course it's not a BSD clone in general and really it's own branch, just with a common origin.

What exactly? Quirky BSD and a WebGUI?
Exactly!
I like what I've read on ZFS, and I like what I've read on FreeNAS. Something to experiment with, not necessarily saying it is the end all be all best thing since sliced bread. Rather, I'm just saying I like what I've read and would like to incorporate it. And I do like the checksumming ZFS offers, and when combined with ECC RAM, it'll let me feel comfortable that the data is good.

Not really. A CPU and a chipset is a CPU and a chipset - the only trick with the SoC is that they're binned parts, and they're throttled. People throw around TDP numbers like they're reflective of power use, but most Intel CPUs come in well under their TDP when under load, and everything since Sandy Bridge idles in single digits. The difference IRL between the Xeon-D SoC and a socketed E3 are likely insignificant next to the power of the dark side... err.... I mean, the amount of power required to keep all your HDDs spinning.

Well yeah it's all throttled, but considering it is a combined platform, the PCH is included in the power calculations. Yes TDP isn't the final say on power consumption, nor is it intended to truly represent power consumption, but for Intel it has historically been a good gauge for consumption when stressed to max.

I'm just saying that a 45W Xeon CPU and motherboard will consume more at idle or load than a 45W Xeon D package, because the assorted controllers are factored into the load already and not an additional element.

http://www.anandtech.com/show/9185/intel-xeon-d-review-performance-per-watt-server-soc-champion/15
That shows idle consumption when comparing against the 45W Xeon D-1540 (8c/16t). The next page shows power consumption under load as a web server.
Don't just look at the graphs, especially on the next page, read what Johan has to say on the matter.

Not saying this demonstrates entirely going to Xeon D for my server, but just to show that there is a difference in power usage, especially when adding controllers to match functionality of what the Xeon D offers on package.

Which is why if I go E3, I am liking the E3-1200 v5 series because the C236 offers nice functionality and hopefully with the right motherboard, I won't have unnecessary components stealing power for no gain. But still much to decide.

BTW, if it seems like we are all getting snippy I think it's because we are ready for you to make a decision and buy something.

Sometimes you gotta either crap or get off the pot!


Y'all are getting rather snippy now. JK

As for the analogy of crapping, well, I'm constipated, so it'll be awhile! :biggrin:
Seriously though, I am no where near ready to buy right now anyway, so prepare for delayed satisfaction. But I'd like to get a near-final component list so I can watch for individual deals and just know my budget goal.
 
Feb 25, 2011
16,823
1,493
126
Really? Damn. Oh well, don't really care in the end. I figured there'd be some similarity between FreeBSD and OS X considering the Darwin kernel's origin. Then again, Darwin evolved from BSD so I don't know what differences were incorporated in FreeBSD in comparison to BSD. And of course they've developed it themselves so of course it's not a BSD clone in general and really it's own branch, just with a common origin.

Yeah - there's been divergence. And Apple rewrote a lot of tools themselves, so all the command syntax is different. And the overall layout of the file system, besides. (Although some of the "old fashioned" stuff like /etc/fstab still works, but it's ignored by default.)

It's still a *nix OS, but there's probably about the same in common between FreeBSD/OSX as there is between FreeBSD/Linux or Linux/OSX.

Well yeah it's all throttled, but considering it is a combined platform, the PCH is included in the power calculations. Yes TDP isn't the final say on power consumption, nor is it intended to truly represent power consumption, but for Intel it has historically been a good gauge for consumption when stressed to max.

I'm just saying that a 45W Xeon CPU and motherboard will consume more at idle or load than a 45W Xeon D package, because the assorted controllers are factored into the load already and not an additional element.

http://www.anandtech.com/show/9185/intel-xeon-d-review-performance-per-watt-server-soc-champion/15
That shows idle consumption when comparing against the 45W Xeon D-1540 (8c/16t). The next page shows power consumption under load as a web server.
Don't just look at the graphs, especially on the next page, read what Johan has to say on the matter.

Not saying this demonstrates entirely going to Xeon D for my server, but just to show that there is a difference in power usage, especially when adding controllers to match functionality of what the Xeon D offers on package.

True. But 20w difference over the course of a year is <$20 with what I pay for electricity. When a Xeon-D board is $1k and a used "good enough" server is $300, there's a cost-benefit calculation that's hard to ignore.
 

XavierMace

Diamond Member
Apr 20, 2013
4,307
450
126
True. But 20w difference over the course of a year is <$20 with what I pay for electricity. When a Xeon-D board is $1k and a used "good enough" server is $300, there's a cost-benefit calculation that's hard to ignore.

Which is what I've been trying to say all along.
 

VirtualLarry

No Lifer
Aug 25, 2001
56,453
10,120
126
True. But 20w difference over the course of a year is <$20 with what I pay for electricity. When a Xeon-D board is $1k and a used "good enough" server is $300, there's a cost-benefit calculation that's hard to ignore.
True, true.
 
Last edited:

destrekor

Lifer
Nov 18, 2005
28,799
359
126
True. But 20w difference over the course of a year is <$20 with what I pay for electricity. When a Xeon-D board is $1k and a used "good enough" server is $300, there's a cost-benefit calculation that's hard to ignore.

Which is what I've been trying to say all along.

Fair point, as 20w isn't at all extreme.

But one thing to point out is only the 8-core Xeon D boards are around $1k. I can get a 6-core model for ~$600ish. And in both single and multithreaded applications, it appears the D-1528 often outperforms the E3-1230 v3 and sometimes even the E3-1275 v5.

This page is for the D-1587, but it is conveniently open in another tab and has the processors I mentioned, and others.

And as for 20w difference, here's the thing, what is that getting me? For a $300ish used server, it's likely going to have a E5xxx or X5xxx CPU or two in there.

https://www.cpubenchmark.net/compare.php?cmp[]=2507&cmp[]=1251&cmp[]=2693

That's comparing a D-1540, E3-1230 v5, and E5640.

I wanted to grab a different Xeon D, but unfortunately, no one has ran anything else through Passmark. Single-thread should be similar for the other models, they are all clocked roughly the same and share the same ratio of cache/core count.

Now, the E5640 is impressively not all that far behind the Xeon D when it comes to single-thread, but passmark shows the E3-1230 v5 relatively smoking the D, which is a little surprising as the other benchmarks aren't showing that extreme of a difference between the two (per servethehome.com), but anandtech's comparison of the D-1540 and the E3-1200s shows roughly the same difference between the E3-1240 v3 and the D-1540. It is close to the Low Power variants of the E3-1200 series but of course those much more similar.

But it is also wise to focus upon total Passmark score, and at Anandtech and STH and other sources, the multi-threaded benchmarks. The Westmere-EP gets destroyed in comparison by both the D-1540 and E3-1230 v5.

The way I figure it, if there's going to be a 20W or higher difference, I want the CPU to have a better perf/watt performance as well, or at least in the same league.

The D-1528 should be at right about the same level as the E3-1230 v5 when comparing multi-threaded performance.

Now I've been looking, and frankly the two would end up at right about the same cost for the package. An E3-1230 v5 and comparable C236 motherboard from Supermicro would come out to be just a little bit cheaper than a D-1528 package.

There are differences in the two packages I'd be comparing here:
X10SDV-6C-TLN4F [D-1528]:
up to 128GB ECC RDIMM or 64GB ECC UDIMM
Dual 1000base-T (via Intel i350-AM2)
Dual 10Gbase-T (via SoC)
2x PCIe 3.0 x8 (one in x16), and 1x PCIe x4 (in x8)
m.2 PCIe 3.0 x4 slot (2242/2280)

X11SSH-LN4F [+ E3-1230 v5]:
up to 64GB ECC UDIMM
Quad 1000base-T (via Intel i210-AT)
same PCIe layout
m.2 PCIe 3.0 x2 (2280)

The X11SSi-LN4F is very similar, except ATX as opposed to the above's microATX form factor. It's drops from 8x SATA3 to 6x SATA3, and uses that to make the m.2 slot PCIe 3.0 x4 as opposed to x2 and offers 2242, 2260, 2280, and 22110 lengths. Roughly the same price.

With the D-1528, I wouldn't actually utilize 10GbE, not worried about that anymore, but it would offer the same 4x gigabit LAN connections which is what I am after.

The E3-1230 v5 might offer more processing headroom in single threaded processes, which could be helpful, but I guess outside of VC-1 transcoding, almost everything else in Plex is multithreaded, which is where I expect the most processing use to come from.

And when comparing the E3 to the Westmere chips, the way I see it, I'd be getting a heck of a lot more performance out of the same wattage. So I could go cheap and get something that might function well enough for everything I need, but I'd expect it to need to remain at higher clocks to compute for far longer than the other two in comparison. So a 20W difference at idle and at full load might be what it is, but how long would the one stay at full load when the other chips to get it back down to idle faster, or might not even need to be a full load to begin with? So that 20W difference becomes a wider gulf when you get down to it.

And I expect the wattage difference to be larger when comparing the Westmere to the E3 CPUs that Anandtech benchmarked.

The one pain in the rear that would come with the E3-1230 v5 is xHCI. Not sure how I would address that.
 
Feb 25, 2011
16,823
1,493
126
Fair point, as 20w isn't at all extreme.

...

And as for 20w difference, here's the thing, what is that getting me? For a $300ish used server, it's likely going to have a E5xxx or X5xxx CPU or two in there.

I was actually comparing it to my system, since I have difficulty seeing past the nose on my face - I paid $130 for a open-box motherboard, and have seen E3-1230v3s going for <$160 on eBay. (Comparing platform to platform.)

Yes, if the whole server with a case, PSU, RAM, etc., is all under $300, you're looking at older, more power-hungry equipment. But even if they idle at +60w, that's still a <$60/year difference.
 
Last edited:
Feb 25, 2011
16,823
1,493
126
And when comparing the E3 to the Westmere chips, the way I see it, I'd be getting a heck of a lot more performance out of the same wattage. So I could go cheap and get something that might function well enough for everything I need, but I'd expect it to need to remain at higher clocks to compute for far longer than the other two in comparison. So a 20W difference at idle and at full load might be what it is, but how long would the one stay at full load when the other chips to get it back down to idle faster, or might not even need to be a full load to begin with? So that 20W difference becomes a wider gulf when you get down to it.

A 2-hour movie will last 2 hours, whether you're transcoding it on a Haswell/Skylake or Westmere. For other computationally intensive tasks, you're correct about the "race-to-idle" thing.
 

Ranulf

Platinum Member
Jul 18, 2001
2,411
1,312
136
Fair point, as 20w isn't at all extreme.

And when comparing the E3 to the Westmere chips, the way I see it, I'd be getting a heck of a lot more performance out of the same wattage. So I could go cheap and get something that might function well enough for everything I need, but I'd expect it to need to remain at higher clocks to compute for far longer than the other two in comparison. So a 20W difference at idle and at full load might be what it is, but how long would the one stay at full load when the other chips to get it back down to idle faster, or might not even need to be a full load to begin with? So that 20W difference becomes a wider gulf when you get down to it.

Have you changed out the light bulbs in your home for LEDs/CFL?
 

destrekor

Lifer
Nov 18, 2005
28,799
359
126
As long as you're shopping for mATX or ATX motherboards, that's not a problem - or even a particularly large expense - either.

www.ebay.com/itm/IBM-Broadcom-NetXt...Ethernet-Server-Adapter-49Y7949-/201520615462

Well of course it's not a big expense, and I'm not necessarily going out of my way to obtain that in the package, but it sure as hell is nice, especially if it's onboard 10GbE because that's also future proof. At the same time, I do hear 10Gbase-T is more power hungry than a standard SFP+ transceiver, and presumably uses more power than a gigabit LAN connection.
But I do like to stick to Intel NICs. I'd get a basic Intel 4-port if I had to, or 2-port if I have 2 onboard ports (or would a 4 port card, and disable the onboard, be more efficient?)

A 2-hour movie will last 2 hours, whether you're transcoding it on a Haswell/Skylake or Westmere. For other computationally intensive tasks, you're correct about the "race-to-idle" thing.

I know that a on-the-fly transcode will of course take the entire time, but the question even exists for that: does it require more or less of the CPU during that whole time? Older CPU = it needs more of it for the same task, thus more energy. It won't be a race to idle, so to speak, but it'll be a case of only using what it needs. A nice newer more efficient chip can get more done with less.

Have you changed out the light bulbs in your home for LEDs/CFL?

I do indeed utilize LED lighting. CFL sucks in my experience.
 

destrekor

Lifer
Nov 18, 2005
28,799
359
126
Not sold on the idea just yet, but this used server has piqued my interest:
http://www.ebay.com/itm/2U-Supermic...sh=item1eab6b6d7b:g:yl8AAOSwYaFWeJov#viTabs_0

edit:
Would likely exceed my desired power budget, especially with Westmere compared to modern efficiency, but, it's slightly enticing. I'm continuing the prowl on ebay so something else, hopefully better, will catch my eye. Might still prefer to build a Xeon E3-1230 v5 or Xeon D-1528 server, but a server like that Supermicro one would cut the cost of the system-build (minus drives) in half.
 
Last edited:

XavierMace

Diamond Member
Apr 20, 2013
4,307
450
126
That's a little bit on the high side for what you're getting. Also, just to make sure you're aware, the L5630's are quad cores, the L5640's are hex cores.
 

destrekor

Lifer
Nov 18, 2005
28,799
359
126
That's a little bit on the high side for what you're getting. Also, just to make sure you're aware, the L5630's are quad cores, the L5640's are hex cores.

Yeah for that generation of hardware I'd rather not pay that high of entry for used, but that was found after broadening my search terms in ebay because I had been seeing only minimal hits. Now I'm seeing more so I'll be paying a little more attention.

I like with that seller/company that you can pay $50 for shipping, OR, you can buy a separate special warranty package that includes expedited shipping, a more thorough test of all components and ports, and a 90 day warranty instead of 30 day. That's basically taking advantage of the people who can't be bothered to read thoroughly. lol

I have found a few other options, and only skimming the surface of what I'm finding now, so I'll be constantly looking to see if there's anything really special that hits.
 

destrekor

Lifer
Nov 18, 2005
28,799
359
126
Did you ever check with Xbyte? Call them as tell them what you want as far as a server goes. They provide real tech support and a real warranty much like Dell, HP, etc...

http://www.xbyte.com/Warranty.aspx

I could check it out, but browsing their site, the prices look to be far far higher, to the point that I couldn't convince myself to buy used when I could assemble something cheaper new.

Sure the prices are great compared to brand new, and their warranty and support factors into the cost, but I wouldn't really get that much more computational capability out of a server I could afford at their site, compared to what I can build with modern chipsets.

But I've only scratched the surface on their site, haven't dug up too much, so I'll keep poking around.
 

destrekor

Lifer
Nov 18, 2005
28,799
359
126
:hmm:
http://www.servermonkey.com/refurbished-hp-proliant-dl180se-g6-12p-configure-to-order.html

Refurbished HP ProLiant DL180se G6 12P (Configure To Order)

Chassis:
1 x HP ProLiant DL180se G6 12-Port Chassis $175
Select Your CPU - Option #1
1 x 2.4 GHz Quad-Core Intel Xeon Processor with 12MB Cache -- E5620 $20
Select Your CPU &#8212; Option #2:
1 x 2.4 GHz Quad-Core Intel Xeon Processor with 12MB Cache -- E5620 $20
Upgrade Your Memory:
1 x 32GB Memory Upgrade Kit (8x4GB) PC3-10600R $50
Choose Your Hard Drive RAID controller:
1 x No RAID $0
Choose Your RAID Level:
1 x No RAID Installation $0
Add a Hard Drive &#8212; Option #1
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #2
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #3
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #4
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #5
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #6
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #7
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #8
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #9
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #10
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #11
1 x Blank Tray $12
Add a Hard Drive &#8212; Option #12
1 x Blank Tray $12
Would you like to add a Network Card?
1 x Included Dual Port Gigabit NICs $0
Please select you Power Supply Unit:
1 x Redundant HP 750 Watt PSU $100
Would you like to add Rails?
1 x No Rails $0
Would you like to add HP Tech Support?
1 x No License $0
Would You Like to Upgrade Your Warranty?
1 x 1 Year ServerMonkey Warranty (Included) $0


Only thing, 2x E5620 has a lower passmark than an E3-1230 v3. ServerMonkey has better support than ebay sellers, so that helps with the price being higher than on ebay.

Tough to settle with that vs a brand new E2-1230 v5 configuration or even that Xeon D-1528, considering both would achieve both high single-threaded speed as well as combined multi-thread speed, all with a heck of a lot less power usage compared to dual-CPU Westmere.

I forgot: I looked up a comparable system on Server Monkey, but here's what piqued my curiosity:
http://www.ebay.com/itm/HP-Proliant...116254?hash=item58d59f695e:g:eqgAAOSwzgRWuKby
 
Last edited:

frowertr

Golden Member
Apr 17, 2010
1,371
41
91
That's why I was asking if you called them. They don't have everything on their site. Give them a budget.
 
sale-70-410-exam    | Exam-200-125-pdf    | we-sale-70-410-exam    | hot-sale-70-410-exam    | Latest-exam-700-603-Dumps    | Dumps-98-363-exams-date    | Certs-200-125-date    | Dumps-300-075-exams-date    | hot-sale-book-C8010-726-book    | Hot-Sale-200-310-Exam    | Exam-Description-200-310-dumps?    | hot-sale-book-200-125-book    | Latest-Updated-300-209-Exam    | Dumps-210-260-exams-date    | Download-200-125-Exam-PDF    | Exam-Description-300-101-dumps    | Certs-300-101-date    | Hot-Sale-300-075-Exam    | Latest-exam-200-125-Dumps    | Exam-Description-200-125-dumps    | Latest-Updated-300-075-Exam    | hot-sale-book-210-260-book    | Dumps-200-901-exams-date    | Certs-200-901-date    | Latest-exam-1Z0-062-Dumps    | Hot-Sale-1Z0-062-Exam    | Certs-CSSLP-date    | 100%-Pass-70-383-Exams    | Latest-JN0-360-real-exam-questions    | 100%-Pass-4A0-100-Real-Exam-Questions    | Dumps-300-135-exams-date    | Passed-200-105-Tech-Exams    | Latest-Updated-200-310-Exam    | Download-300-070-Exam-PDF    | Hot-Sale-JN0-360-Exam    | 100%-Pass-JN0-360-Exams    | 100%-Pass-JN0-360-Real-Exam-Questions    | Dumps-JN0-360-exams-date    | Exam-Description-1Z0-876-dumps    | Latest-exam-1Z0-876-Dumps    | Dumps-HPE0-Y53-exams-date    | 2017-Latest-HPE0-Y53-Exam    | 100%-Pass-HPE0-Y53-Real-Exam-Questions    | Pass-4A0-100-Exam    | Latest-4A0-100-Questions    | Dumps-98-365-exams-date    | 2017-Latest-98-365-Exam    | 100%-Pass-VCS-254-Exams    | 2017-Latest-VCS-273-Exam    | Dumps-200-355-exams-date    | 2017-Latest-300-320-Exam    | Pass-300-101-Exam    | 100%-Pass-300-115-Exams    |
http://www.portvapes.co.uk/    | http://www.portvapes.co.uk/    |