Server hardware is a ripoff

[jlee]

Oh come on. I work at a smallish place and could get by on one server. However it would be a tricked out R730xd 768GB RAM, 32 core VM monster.

You'd be better off running two lesser spec'd machines and setting up HA/DRS.

 

[ViviTheMage]

Oh come on. I work at a smallish place and could get by on one server. However it would be a tricked out R730xd 768GB RAM, 32 core VM monster.

too many eggs in one basket, at least divide that sucker in two.

 

[poofyhairguy]

You'd be better off running two lesser spec'd machines

I have often wondered that. With servers costing more than twice as much as consumer level tech, for light use what is the real benefit over a single super server vs two cheapo desktops like OP bought with some sort of software RAID that rsyncs with the identical box every hour or whatever? I mean sure ECC RAM and PSU redundancy is great, but it seems for every bit of protection a real server can give you complete system redundancy would still be better. Especially because the clone server could be in the owner's basement or some satellite office, which then makes your setup fire proof.

I mean I get why companies that user servers to serve data externally go for real serves, but for an office file server isn't two better than one of anything?

 

[ViviTheMage]

I have often wondered that. With servers costing more than twice as much as consumer level tech, for light use what is the real benefit over a single super server vs two cheapo desktops like OP bought with some sort of software RAID that rsyncs with the identical box every hour or whatever? I mean sure ECC RAM and PSU redundancy is great, but it seems for every bit of protection a real server can give you complete system redundancy would still be better. Especially because the clone server could be in the owner's basement or some satellite office, which then makes your setup fire proof.

I mean I get why companies that user servers to serve data externally go for real serves, but for an office file server isn't two better than one of anything?

Literally one word, support. If a piece of hardware fails and it's indeed mission critical/costs the company money to be down, you want that ah heck supported. Even if you had a cowboy style setup to DR/HA, what happens if that person who set it up to leave. You could end up paying them way more then they're worth to keep them around.

 

[sourceninja]

...

 

[Red Squirrel]

I can understand ECC ram and such being more expensive and good idea to have but when it comes to the OEMs they do overcharge WAY too much. For example, go to Dell's website and build a server. The base price is actually really reasonable if you only want something with a 120GB drive and 1GB of ram and a Celeron(why is that even an option?!?). Add 4GB of ram and maybe bump up the cpu to a higher end(lol) Celeron, and suddenly it's practically double the price. For that price I can build a Supermicro based server with like 64GB of ECC ram and a Xeon processor and SSD for OS drive. Best part is, the parts like hard drives are standard so I can always buy more elsewhere and add them in. If I was owner of a company I'd rather build 2 or even 3 of the custom built Supermicro servers and build more redundancy into the system, than pay an arm and a leg just for the 4 hour support.

The 4 hour support *IS* nice, but your environment should never be designed in such a way where it even matters. You should have enough redundancy built into your systems that if a server goes down it's not really a huge deal. You want to get on it ASAP before another goes down, but your actual system is still running fine.

Some companies like Google actually have this right. Most of their gear is actually consumer grade and/or custom. It's node based, if a node goes down it's not a big deal. Fix it and put it back in service. Nobody even notices. If I was a company owner I rather have IT staff that is physically building and fixing stuff, than spending hours on the phone with Dell support trying to convince some guy in India that the hard drive really is dead and that a new one is needed, then having to deal with the fact that they sent the wrong part etc. With custom stuff you'd be able to use standard hard drive (using that as an example but could be ram or anything else) and have spares or easily be able to get new parts. 3 years down the line you're not dead in the water either. With OEM stuff it's all proprietary and once the warranty runs out you're on your own because good luck finding parts. You can go through ebay but do you really want to trust that.

Then there's the whole liability thing, which I find is completely retarded and babyish, but one of the main reasons companies do go with OEMs is so they can put the blame on someone else when something goes wrong. Stop the stupid babyish blame game and just setup something that works for a decent price and when it does break deal with it like an adult and move on.

 

[XavierMace]

I can understand ECC ram and such being more expensive and good idea to have but when it comes to the OEMs they do overcharge WAY too much. For example, go to Dell's website and build a server. The base price is actually really reasonable if you only want something with a 120GB drive and 1GB of ram and a Celeron(why is that even an option?!?). Add 4GB of ram and maybe bump up the cpu to a higher end(lol) Celeron, and suddenly it's practically double the price. For that price I can build a Supermicro based server with like 64GB of ECC ram and a Xeon processor and SSD for OS drive. Best part is, the parts like hard drives are standard so I can always buy more elsewhere and add them in. If I was owner of a company I'd rather build 2 or even 3 of the custom built Supermicro servers and build more redundancy into the system, than pay an arm and a leg just for the 4 hour support.

I've never found Supermicro to be too much cheaper than Dell/HP. I would love to see you quote some actual numbers. There's nothing stopping you from putting standard parts into a Dell/HP. I've got OEM Hynix RAM in my HP's, tossed the old HP memory in some Dell's, and put the Dell memory in some Supermicro's. I've got similarly mix and matched drives. Are there some proprietary parts? Sure, but Supermicro has proprietary parts too.

Some companies like Google actually have this right. Most of their gear is actually consumer grade and/or custom. It's node based, if a node goes down it's not a big deal. Fix it and put it back in service. Nobody even notices.

ROFL. That's not even remotely practical for smaller companies. Everyone here seems to be operating under the assumption that all companies have a complete IT staff onsite. Setting up a cluster, especially a multisite cluster isn't something there's an easy button for.

Not to mention I've worked for a company that used that system and it's got no shortage of headaches as well.

If I was a company owner I rather have IT staff that is physically building and fixing stuff, than spending hours on the phone with Dell support trying to convince some guy in India that the hard drive really is dead and that a new one is needed, then having to deal with the fact that they sent the wrong part etc.

Until you realize you are paying an IT Admin $80k a year plus benefits to sit there waiting for something to break rather than paying Dell/HP/SMS/etc like $800/year.

I can call HP (or email them) and in 30 minutes I'll have a new drive on the way. SMS will have it on the way in like 5 minutes.

With custom stuff you'd be able to use standard hard drive (using that as an example but could be ram or anything else) and have spares or easily be able to get new parts. 3 years down the line you're not dead in the water either. With OEM stuff it's all proprietary and once the warranty runs out you're on your own because good luck finding parts. You can go through ebay but do you really want to trust that.

There's nothing stopping you from doing the same thing with an OEM. Have you ever actually done this before? I can get parts for my HP/Dell's just as easy as my Supermicro's.

If you're outside warranty and still using the box (that's a whole different conversation) there's companies like SMS (System Maintenance Services) that offer post manufacturer warranty support.

Then there's the whole liability thing, which I find is completely retarded and babyish, but one of the main reasons companies do go with OEMs is so they can put the blame on someone else when something goes wrong. Stop the stupid babyish blame game and just setup something that works for a decent price and when it does break deal with it like an adult and move on.

I've never seen that used as the primary factor in the purchasing process. Not to mention it doesn't matter how much you spend, something can still go wrong.

Not to mention all that assumed you keep a full IT staff onsite. We support roughly ~110 customers. All of their equipment is configured to send SNMP trap alerts to us. We get an alert, call HP/Dell/SMS, tell them they need to go replace a drive/fan/battery/whatever. Call the client and tell them somebody will be there at X time and will need access to the server closet. Our client has 0 IT staff onsite and doesn't even have to touch the server.

 

[ch33zw1z]

"server hardware is a ripoff" - says naive individuals until their critical business functions are in the crapper. Then they find someone to cry to and try to get it repaired cheaply too.

You can have it good, fast, or cheap...but you can only pick 2.

@XavierMace, that applies to TPM's like SMS as well. In most cases they'll provide adequate support for a comparatively reasonable price(largely cheaper than other vendors). But there's a handful of cases in the last few years where I've had to pick up the pieces after them. It's worth noting that these incidents weren't on x86 systems.

 

[child of wonder]

Nothing like getting a call from a Director of IT who is in dire straights and needs help. Coming onsite we find whitebox servers in disarray, a hackneyed storage array running some version of Linux or ZFS, and everything being managed by a litany of custom scripts.

Why the troubles? The guy who built it all either left or did it wrong. His attempts to save the company money put them at risk and now they're having to pony up $200/hr for an expert or experts to come fix it.

Why buy server hardware from Dell, HP, Cisco, etc.? Why buy storage from Netapp, EMC, Tintri, HP, etc.? Because a large part of business is mitigating risk. Purchasing these tested, solid solutions reduces risk for the business and, more importantly, for the IT staff themselves. When a business is losing thousands or more dollars a minute during downtime, that $10,000 in savings by running everything on 1337 PC gear doesn't seem so significant anymore which is why big companies make the investment in enterprise grade hardware and software.

Do you really think all these industry experts, from industry CIOs, CFOs, research firms, etc. are so stupid for buying enterprise gear when little old you has it all figured out that your desktop Intel CPU and Gigabyte motherboard from Microcenter is really the way to go? Do you really think they haven't run the numbers, weighed the pros and cons, and analyzed the risks then decided to buy the needlessly more expensive gear anyway?

Know what happens to IT guys who build all this whitebox crap then need to call in the pros when shit goes south? They get replaced with people who want to actually act like professionals and not put the business and themselves at risk by saving a few thousand dollars with desktop gear to run their business critical applications.

This is the same reason I didn't pour the foundation for my house, do all the electrical work myself, tape, seal, and mud the drywall, etc. No matter what, I can't do it as well as the pros who do it day in and out for a living and know what they're doing. Would I have a house? Sure I would; a dilapidated, ugly, crumbling, piece of shit house that is entirely my responsibility to keep standing even as it falls around me and when it does eventually fall, it's my fault. This is why I pay, for example, $200k for a house that is actually $50k in building materials -- because the risk isn't worth it and my family needs a solid roof over their heads.

I will say, however, that whitebox or desktop gear makes sense for small mom and pop stores who can't even afford an annual $1,000 IT budget. I did it myself for a tiny company I did work for back in 2004. In that scenario, the risk must be taken on because the money isn't there to mitigate it.

 

[Riverhound777]

Something I haven't seen mentioned here (Haven't looked too close) is a happy medium. Being an IT consultant myself, I often buy servers for clients. The thing we do is buy the HP server base model we need, and buy the parts separately. We get the server itself from Ingram Micro for much less than HP sells it for, and we buy HP brand parts like drives, ram, second CPU, second PS from other sites and install them ourselves. They are still covered under warranty, and we save a ton of money over building up a system through HP direct. Of course you need to be a re-seller to do this, but just a thought.

 

[moonbogg]

"server hardware is a ripoff" - says naive individuals until their critical business functions are in the crapper. Then they find someone to cry to and try to get it repaired cheaply too.

You can have it good, fast, or cheap...but you can only pick 2.

I went with good & cheap. Thanks.

 

[ch33zw1z]

So you bought a 10 year old server off ebay?

 

[Ns1]

txt

a well thought out post, who would've thunk it.

 

[moonbogg]

a well thought out post, who would've thunk it.

I'm sure it was brilliant, but its behind a pay wall.

 

[XavierMace]

@XavierMace, that applies to TPM's like SMS as well. In most cases they'll provide adequate support for a comparatively reasonable price(largely cheaper than other vendors). But there's a handful of cases in the last few years where I've had to pick up the pieces after them. It's worth noting that these incidents weren't on x86 systems.

I would never say they're flawless. But in my experience they are consistently better than Dell/HP direct and we deal with them a LOT.

 

[xSauronx]

Something I haven't seen mentioned here (Haven't looked too close) is a happy medium. Being an IT consultant myself, I often buy servers for clients. The thing we do is buy the HP server base model we need, and buy the parts separately. We get the server itself from Ingram Micro for much less than HP sells it for, and we buy HP brand parts like drives, ram, second CPU, second PS from other sites and install them ourselves. They are still covered under warranty, and we save a ton of money over building up a system through HP direct. Of course you need to be a re-seller to do this, but just a thought.

is the support as straightforward as buying a packaged system at once? that is, i am used to buying a ready to go server from dell or hp, and then if there is a problem i just open a ticket with the serial # and problem and i am good to go.

if you have a problem with your sort of custom system, is it still that easy or do you have to track the part numbers/sn for support, too?

 

[ch33zw1z]

xSauronx: x86 vendors like Dell/HP/IBM(now Lenovo for x86) won't care. If your box is on maintenance or warranty, and the parts that need replacing are valid part numbers for the box(check their sites for support parts listing), then they'll be ok with it. HP uses "spare" part numbers to identify valid parts, IBM/Lenovo use FRU or CRU numbers. I don't know what Dell uses...almost never touch a Dell..

What you're talking about won't happen until you hit enterprise type gear. Then companies start to get picky about what's in the box.

 

[ch33zw1z]

I would never say they're flawless. But in my experience they are consistently better than Dell/HP direct and we deal with them a LOT.

Cool. We're also in different geographies. You may have better field techs out there too.

 

[JM Aggie08]

You get what you pay for.

Use a managed service if you do not want to purchase legitimate hardware.

 

[Riverhound777]

xSauronx: x86 vendors like Dell/HP/IBM(now Lenovo for x86) won't care. If your box is on maintenance or warranty, and the parts that need replacing are valid part numbers for the box(check their sites for support parts listing), then they'll be ok with it. HP uses "spare" part numbers to identify valid parts, IBM/Lenovo use FRU or CRU numbers. I don't know what Dell uses...almost never touch a Dell..

What you're talking about won't happen until you hit enterprise type gear. Then companies start to get picky about what's in the box.

This is correct, as long as the part you're having issues with is HP branded and compatible with the server in question, they'll support it. They don't care if you paid $600 for the drive direct from HP or if you paid $200 for a bulk HP spare part drive. Seriously, the difference can be that much.

Now keep in mind there are a lot of fake spare part drives and such out there, so you do have to be mindful. Best way is to read site reviews and keep a list of vendors you've used that are good. Shipping speed can also vary greatly. We use a site called Allhdd.com but they are slow and pretty bad customer service. If we need the parts fast, we go elsewhere and spend a bit more.

Not sure why i'm sharing all the secrets, but here it is, for HP at least. Find the server model you need, such as DL380P Gen 8. Google DL380P Gen8 quickspecs to find the PDF file on HP's website. Look in there and find both the "Smartbuy" model of the server you want, and all the part numbers for the drives, ram, PS, whatever. Then google or search for those parts on the various websites, IM, CDW, Serversupply, TechData, etc. Determine the best deal for the whole package, and there ya go. Also note for second CPUs you want the Kit, that comes with the heatsink and fan, not just the CPU.

 

[Scarpozzi]

The servers I ran recently were quad processor, 10+ core per processor systems. Running Red Hat on those, they would spin up 80 virtual processors.....threads didn't stand a chance...that thing screamed compared to the Sun hardware it replaced.

 

[ch33zw1z]

Not sure why i'm sharing all the secrets, but here it is, for HP at least.

Just can't help ourselves. It's a painful learning process. Also, good additional info.

 

[Scarpozzi]

I have often wondered that. With servers costing more than twice as much as consumer level tech, for light use what is the real benefit over a single super server vs two cheapo desktops like OP bought with some sort of software RAID that rsyncs with the identical box every hour or whatever? I mean sure ECC RAM and PSU redundancy is great, but it seems for every bit of protection a real server can give you complete system redundancy would still be better. Especially because the clone server could be in the owner's basement or some satellite office, which then makes your setup fire proof.

I mean I get why companies that user servers to serve data externally go for real serves, but for an office file server isn't two better than one of anything?

When dealing with office file servers, the key is to run a responsible level of RAID and to spread your data across enough spindles to handle your user load. That doesn't matter if you're running it on a SAN, NAS, or Direct Attached array... These days, a lot of the 2.5" form factor disks can be sandwiched in a 19" rack to provide 24-25 disks in an array (horizontal configuration). You can do RAID6 or RAID10 to provide a little cushion. New SANs will do block level tiering to speed up reads/writes and move stagnant data to SATA from SCSI or SSDs. Your best bet is to decide what you can afford and if you can't go big, just provide yourself room to expand if you need it.

Direct Attached isn't bad, but you need to manage the arrays efficiently for scalability, max users, and backups. I've done a few different flavors of DFS in these kinds of systems to make multiple smaller arrays look like one big disk...and that was just so I could break the data into smaller manageable chunks for backups.

It can be a mess if you don't approach it right, but these days a lot of firms are moving away from tape backups and are doing more snaps and backup to disk. Because of the data I've always dealt with, the data was saved to tape, packaged, and shipped off site weekly to be stored in a vault. Disaster recovery plans sometimes rule what technologies you can adopt too....

 

[MiniDoom]

You'd be better off running two lesser spec'd machines and setting up HA/DRS.

This would also require the purchase of vcenter which no small office is going to do.

 

[dougp]

This would also require the purchase of vcenter which no small office is going to do.

That assumption is using VMWare - they could use XenCenter or Hyper-V to provide failover, or even using Veem with free VMWare.