Budget Server Options (Cheap Cores on a Budget through Ebay)

[tential]

I saw that as well. I think ES do work though right? I would say that's more of a "Keep it in mind!!!" but not a "legit price" to say for sure.
I think Haswell is more expensive that it puts it in direct prirces with new threadripper systems, although you get the increased IPC of Haswell, but I was seeing the most legitimate options at $700 used. And that's still a lower clockspeed than Threadripper so it may be more equal for my gaming purposes.

I really think Ivybridge is the sweetspot though for prices for used from what I've seen.

My primary use for this is a "gaming server" that serves games around the home (and handles a few other tiny tasks but those only need 2-4 cores).

So keep that in mind, although feel free to let anyone know why you like a processor. Hopefully people can use this thread to find budget server solutions for themselves for whatever their use case is.

So anything that beats out the 2P Ivybridge Xeon setups in price/performance/core count for my gaming server purposes? Assuming I'm getting the 2P Ivybridge Xeon/Ram/Mobo for ~$700 (Don't care about other parts).

 

[Markfw]

For 2P Xeon, you would be better off with threadripper. The case is special, and expensive. The you need 8 sticks of ram, then you need a special power supply.

 

[tential]

For 2P Xeon, you would be better off with threadripper. The case is special, and expensive. The you need 8 sticks of ram, then you need a special power supply.

The case is special and expensive, and it's a moot cost between both systems. Either way, I'm buying a special 24 bay hotswap case. It's a sunk cost between both builds, unless the 2p xeon server system includes said case, which would be lucky for me, but I'm not even trying to get that lucky. It's just one more way the used Xeon system could come ahead though, since if I did get the 24 bay hotswap case it saves me a couple hundred dollars.

https://www.ebay.com/itm/Dual-Intel...829362?hash=item3d583453b2:g:hNkAAOSwbF1aHJis

That actually looks like it has everything I need. It's Sandybridge though.

Regarding 8 sticks of ram, why do I need that many? I see that in the ebay listing I noted as well.

How is it regarding finding coolers that work with Xeon Processors/Mobos?

Or a special power supply? Is that required? I just use normal ones as always.

 

[Markfw]

First, 2011-3 is quad channel, and with 2 processors, thats 8 (they each need their own memory)
Coolers, its just socket 2011-3. This is what is on all of my E5's https://www.newegg.com/Product/Product.aspx?Item=N82E16835103202 (but with dual's they may not fit, they are just inexpensive and cool well)

Power supply. It needs at LEAST 2 8 pin plugs, check the motherboard carefully and make sure the PSU you have/get will supply it. With a 2P system, you are also going to draw a lot of juice.

As for why not threadripper, between meltdown and the lack of PCI-E lanes, I would have thought threadripper would be much better for IO as well, but.....whatever

 

[tential]

First, 2011-3 is quad channel, and with 2 processors, thats 8 (they each need their own memory)
Coolers, its just socket 2011-3. This is what is on all of my E5's https://www.newegg.com/Product/Product.aspx?Item=N82E16835103202 (but with dual's they may not fit, they are just inexpensive and cool well)

Power supply. It needs at LEAST 2 8 pin plugs, check the motherboard carefully and make sure the PSU you have/get will supply it. With a 2P system, you are also going to draw a lot of juice.

I see, ya that's nice. I was just making sure I didn't need some exotic solution. If I can fit noctua coolers, I'll go noctua.

As for why not threadripper, between meltdown and the lack of PCI-E lanes, I would have thought threadripper would be much better for IO as well, but.....whatever

IO for sure that's what I'm thinking. PCIE Lanes? I didn't know that please elaborate.

And for Meltdown... without any concrete performance numbers on meltdown/spectre, I feel a lot of the debate is highly speculative on that matter.

Also, I came across this:
https://www.techspot.com/review/1155-affordable-dual-xeon-pc/page7.html

Benches like this:

The E5 really does quite well in gaming for my purposes of being "good enough" for cheap. For ~$100 that's a steal. I think I may build one of these for cheap just to get my foot in the door.

As for the increased IO, I mean so far it's the M2 sata slots. Anything else really that I need regarding mobo features? And it's not just me by the way. Someone may read this thread, have different needs, and go with another solution.

Look at this deal from aliexpress with everything included for $280. What do you guys think?
https://www.aliexpress.com/item/HNa...ea8c807&transAbTest=ae803_4&priceBeautifyAB=0
Mobo+CPU+Ram for $280 seems like a great deal.

 

[tential]

What about this from aliexpress?
This looks too good to be true but I kind of just want to try it...
https://www.aliexpress.com/item/HUA...74c9634&transAbTest=ae803_4&priceBeautifyAB=0

Edit: Good reports from the other chinese mobo from aliexpress posted here on reddit:
https://www.reddit.com/r/pcmasterra...truth_about_ebayaliexpresstaobao_chinese_x79/

 

[Markfw]

The memory is 1333 DDR3. So the entire platform is dead end. And the shipping is 19-39 days.

I certainly would never go that route.

As for the IO, if you read the meltdown/spectre thread, the IO is reduced my like 30% on SSD's M.2, and sata is also affected. ONLY ON Intel

And the 128 PCI-E lanes ? well for one, you could put 3 one TB M.2 drives in there, and thats a LOT of fast storage.

 

[tential]

The memory is 1333 DDR3. So the entire platform is dead end. And the shipping is 19-39 days.

I certainly would never go that route.

40 day shipping time isn't a problem. This isn't a "I need it yesterday" system. I'm a very patient person.
DDR3 ram has been part of the thread the whole time so I don't think it's a deal breaker now 3 pages in.

]
As for the IO, if you read the meltdown/spectre thread, the IO is reduced my like 30% on SSD's M.2, and sata is also affected. ONLY ON Intel

By like 30%?
Any hard numbers/reviews?

And the 128 PCI-E lanes ? well for one, you could put 3 one TB M.2 drives in there, and thats a LOT of fast storage.

Well for me, I need 2 GPU support, support for 16-24 HDDs (I need to use 8 HDD expander cards. I understand there is something regarding drive management and PCIe lanes? At least with my current mobo, I was told I needed to use the x16 slot. Something with DWM I believe? I can't recall I'll have to dig it up).

https://www.amazon.com/SAS9211-8I-8PORT-Int-Sata-Pcie/dp/B002RL8I7M

I need about 2 of those as well which only gets me to 24 drives, unless I just use this for my 15 bay server.

Am I PCI Lane Constrained really in either platform?

 

[Markfw]

40 day shipping time isn't a problem. This isn't a "I need it yesterday" system. I'm a very patient person.
DDR3 ram has been part of the thread the whole time so I don't think it's a deal breaker now 3 pages in.

By like 30%?
Any hard numbers/reviews?


Well for me, I need 2 GPU support, support for 16-24 HDDs (I need to use 8 HDD expander cards. I understand there is something regarding drive management and PCIe lanes? At least with my current mobo, I was told I needed to use the x16 slot. Something with DWM I believe? I can't recall I'll have to dig it up).

https://www.amazon.com/SAS9211-8I-8PORT-Int-Sata-Pcie/dp/B002RL8I7M

I need about 2 of those as well which only gets me to 24 drives, unless I just use this for my 15 bay server.

Am I PCI Lane Constrained really in either platform?

You may have to post in the threadripper builders thread with some of these questions. As to the performance numbers they are in the meltdown thread.

 

[tential]

Could you please link the article/review in this thread? I really want to make sure we source everything and I'd definitely want to read the source of this 30% slowdown on IO performance to understand what I'm getting into.

Which questions would I need to ask in the threadripper build thread specifically that those users would be able to help me?

 

[Markfw]

For starters: https://www.youtube.com/watch?v=JbhKUjPRk5Q

I don't have the time to find the rest of the links. Also see if its on Anandtechs articles ??

As to the questions, since I have never dealt with that many hard drives before, I don't know, just a gut feel.

 

[IEC]

Could you please link the article/review in this thread? I really want to make sure we source everything and I'd definitely want to read the source of this 30% slowdown on IO performance to understand what I'm getting into.

Which questions would I need to ask in the threadripper build thread specifically that those users would be able to help me?

You specifically mentioned unRAID so you could look for "meltdown performance unraid" or similar search string to get an idea.

Here's an example from reddit:
https://www.reddit.com/r/unRAID/comments/7sod1u/meltdown_is_causing_a_meltdown/

People have been downgrading and disabling the fixes due to the performance hit (patch side is right half of the Grafana display):

 

[tential]

You specifically mentioned unRAID so you could look for "meltdown performance unraid" or similar search string to get an idea.

Here's an example from reddit:
https://www.reddit.com/r/unRAID/comments/7sod1u/meltdown_is_causing_a_meltdown/

People have been downgrading and disabling the fixes due to the performance hit (patch side is right half of the Grafana display):

Thanks a lot for that.
So it seems you can just disable the Spectre/Meltdown Fix if it is effecting your workload. According to the thread, he disabled it and it went to normal.
According to the unraid thread, it seems that the 30% is in fact a synthetic benchmark that hasn't been reproduced in the real world, although where it comes from I can't seem to find from that thread/links provided
I feel like I should be expecting at least 10% slow down though judging from that link/threads attached.

Interesting to note, but not much we can do with the anecodtal stuff about Spectre/Meltdown. I'll keep it in mind to watch for an article detailing the issue to see how if I need

For starters: https://www.youtube.com/watch?v=JbhKUjPRk5Q

I don't have the time to find the rest of the links. Also see if its on Anandtechs articles ??

As to the questions, since I have never dealt with that many hard drives before, I don't know, just a gut feel.

Good point. Ya, the issues I ran into were weird ones that I had no idea existed until I posted my build and people explained I was going to have reduced performance due to how I had my PCI E Cards set up. So I am interested in getting to the root of the I/O performance issues I may encounter.

I'm not having any issues with my current unraid 6.4 setup, CPU usage, and dockers though so I wish there was more concrete testing like what you provided me to look at so I can isolate what's going on. I'll have to really look at those SSD numbers specifically as the gaming benches were of course not really effected.

Thanks for that, because if the SSD performance is still acceptable I'm ok. I'll keep this in mind though as they continue to apply patches down the line. Who knows what the performance game looks like 1-2 months from now.

My current searches have been on trying to find people who also bought xeons from aliexpress, but so far it seems like that is a pretty safe bet to do actually.
.

 

[Markfw]

Yes you can uninstall patches and get the performance back, but on a SERVER ? That's an invite to anyone to steal passwords.
I would never do that, but whatever.....

 

[IEC]

Thanks a lot for that.
So it seems you can just disable the Spectre/Meltdown Fix if it is effecting your workload. According to the thread, he disabled it and it went to normal.
According to the unraid thread, it seems that the 30% is in fact a synthetic benchmark that hasn't been reproduced in the real world, although where it comes from I can't seem to find from that thread/links provided
I feel like I should be expecting at least 10% slow down though judging from that link/threads attached.

Interesting to note, but not much we can do with the anecodtal stuff about Spectre/Meltdown. I'll keep it in mind to watch for an article detailing the issue to see how if I need

Good point. Ya, the issues I ran into were weird ones that I had no idea existed until I posted my build and people explained I was going to have reduced performance due to how I had my PCI E Cards set up. So I am interested in getting to the root of the I/O performance issues I may encounter.

I'm not having any issues with my current unraid 6.4 setup, CPU usage, and dockers though so I wish there was more concrete testing like what you provided me to look at so I can isolate what's going on. I'll have to really look at those SSD numbers specifically as the gaming benches were of course not really effected.

Thanks for that, because if the SSD performance is still acceptable I'm ok. I'll keep this in mind though as they continue to apply patches down the line. Who knows what the performance game looks like 1-2 months from now.

My current searches have been on trying to find people who also bought xeons from aliexpress, but so far it seems like that is a pretty safe bet to do actually.
.

Using the flag to disable the Meltdown fix is intended to be used with processors not vulnerable to Meltdown (e.g. Ryzen/TR/Epyc). You can indeed determine that the performance impact (in this user's case, far more than 30% impact) may be too steep for your use case and sacrifice security for performance, or you may find that disabling the fix is not required for your workload.

As far as "anecdotal" reports go, my own benchmarking pre and post fix (UEFI + Windows 10 patch) on Coffee Lake yielded the following:

i7-8700K on Z370 Taichi + 960 EVO 250GB, Windows patch installed, no UEFI/microcode update (UEFI 1.30):
Random Read 4KiB (Q= 32,T= 1) : 514.705 MB/s [ 125660.4 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 687.330 MB/s [ 167805.2 IOPS]
Random Read 4KiB (Q= 1,T= 1) : 43.424 MB/s [ 10601.6 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 156.527 MB/s [ 38214.6 IOPS]

i7-8700K on Z370 Taichi + 960 EVO 250GB, Windows patch installed, UEFI/microcode update installed (UEFI 1.40):
Random Read 4KiB (Q= 32,T= 1) : 305.985 MB/s [ 74703.4 IOPS] -40.6% IOPS
Random Write 4KiB (Q= 32,T= 1) : 413.083 MB/s [ 100850.3 IOPS] -39.9% IOPS
Random Read 4KiB (Q= 1,T= 1) : 43.245 MB/s [ 10557.9 IOPS] within MOE
Random Write 4KiB (Q= 1,T= 1) : 121.128 MB/s [ 29572.3 IOPS] -22.6% IOPS

Red Hat did bare metal benchmarks and found a performance impact of 1%-20% (behind paywall/customer login), with the more I/O heavy workloads being in the 8%-20% range. However, they noted the following about VM performance:

We expect the impact on applications deployed in virtual guests to be higher than bare metal due to the increased frequency of user-to-kernel transitions.

For some real world workloads, the impact was actually worse.

Riak and Redis 50% and 40% performance impact respectively:
https://twitter.com/apolloFER/status/949589014977400833

A certain EHR vendor posted preliminary patch performance testing results on their customer documentation portal this past week. Last update: 1/30 for those of you who may have access...

Per terms, I am not allowed to screenshot or share the document, or even reproduce the text of the document here. However, if this preliminary data is borne out by further testing, this is going to be extremely painful. The previous estimated 30% is nowhere near the worst case using common customer configurations. Skylake and newer is impacted less...

 

[tential]

Unraid also got a new update!
https://lime-technology.com/forums/topic/68985-unraid-os-version-641-stable-release-available/

Just yesterday, Threadripper GPU Passthrough is working.

Unraid + AMD has a lot of outdated posts regarding it since there have been so many updates/fixes.

@Markfw I think in a more serious server certainly, but for a home server application, I'm not worried.

@IEC
Ya, It's definitely something I'm going to keep an eye on. Especially as Ryzen/Threadripper has gotten better and better with unraid, it looks like this is a worse and worse time to buy a system from every single aspect/angle as things are still shaking out daily regarding how these processors work.

If I look back through unraid forums, Ryzen/Threadripper users are having quite the time for some circumstances, but I doubt I'll have any of those growing pains as they should all be patched by now and working well.

I'll be watching the Threadripper 1950x firesales too more and more now as TR2 comes up. With only 10% expected from TR2, the price cut is way more than that 10%, it's a great deal to hop on TR in comparison to TR2 from a price point.

It seems I can't really use any benchmarks regarding Xeon/Intel processors since they are all before this patch was installed. Basically leaves me with a lot of "no information until this is retested!!!"

 

[dlerious]

you have never worked with a 1U chasis to be thinking this naively... lol...
meaning... GOOD LUCK.... 1U's secondary names are called the screaming pizza box case for a reason.

Here's an example at about 7:06 in. https://www.youtube.com/watch?v=yzljmEWXoKg

 

[tential]

Here's an example at about 7:06 in. https://www.youtube.com/watch?v=yzljmEWXoKg

While this is an example of a loud 1U chassis, that doesn't mean a 1U chassis has to be loud.
No one forced them to use the stock fans.

As I already explained, I have no intention of using a 1U chassis. If I did, I would gut the chassis, and replace the fans as I have done in my current 1U configuration, which is completely silent.

As a PC user with the infinite flexibility of the platform, if you don't like the noise level, do something about it. We're all DIY users here, we don't just sit and live with what we're given out the box.

 

[tential]

After a lot more research on this, it looks like you can pick up cheap xeons at ridiculous levels, it really just is a glut of cheap xeon processors out there that it's no wonder they're so popular for server builds and budget gaming builds.

I'm finding Xeon Pairs now for $350 where before I was struggling to find even the single processor for $400.
When you can get 20 cores of Ivybridge for $350-450 off ebay, it's hard to ignore that value.

Found a good subreddit for these types of builds, and for the additional $1k+ I'd have to spend to get threadripper, the benefits just don't justify the cost. It's not really a fair comparison used vs new either but well, the used value of the Xeon is too much to ignore!

 

[tential]

https://www.youtube.com/watch?v=wqv_6qhbHxw
This video extremely changes things. 7900x losing out to Sandybridge in some cases. Makes going Xeon that much more attractive!

Great gaming review of the E5-2680
Had no idea the 7900x had so much architecture changes that really change how it works for gaming.

 

[XavierMace]

https://www.youtube.com/watch?v=wqv_6qhbHxw
This video extremely changes things. 7900x losing out to Sandybridge in some cases. Makes going Xeon that much more attractive!

Great gaming review of the E5-2680
Had no idea the 7900x had so much architecture changes that really change how it works for gaming.

I'm suspicious of those results. While I love my Ivy Bridge EP's, I really don't see the change in the cache setup explaining a processor with higher IPC, higher clock speeds, more memory bandwidth, and more cores losing in these benchmarks.