Blade Centers

[OverWorkedTech]

I am in the beginning stages of looking into blade servers as a possibility in our data center. I am looking into both HP c7000 and IBM's solutions. My ultimate goal would be to have a chassis setup and virtualized with VM ware or Xen.

Is anyone running in a configuration similar to this that can make me aware of possible downfalls and or problems that they have run into?

I have no illusions of moving the bulk of my data center onto this. I am looking at it more as a starting point.

Thanks for any input.

 

[tomt4535]

One of my counterparts at work has 6 c7000 Enclosures pretty well populated with all diffrent kinds of blades and interconnects for his lab. They are really sweet products. Fitting 16 servers into 10U is really great if you are tight on space. The virtual connect interconnect modules make it so much easier to connect all th blades to a SAN or LAN. The fans on it are so quiet and really keep it cool. You can for sure get hundreds of VM's on a single one of those enclosures. You can manage the entire enclosure remotely from your desk, over VPN or whatever, no need to be in front of the servers at all. With ILO, you can power up and power down the servers if they crash or you accidentally shut them off. Virtual media is cool too.

The only thing with the blades is the limited storage. Each half height blade can only fit 2 drives, and the full heights 4. You can get the drive expansion and NAS blades, but they dont offer much space. The best bet is to connect it to a SAN or other shared storage, especially for VMware. If you have a SAN with vmware, you can use VMotion and DRS, HA and all the cool features like that. I havent done anything with the IBM Blades, but the c7000's are pretty nice products

 

[OverWorkedTech]

Tomt4535,

Thanks for your response.
We did plan on putting storage on a separate server. Rack space is not an issue for us, I have plenty of available racks, and rack space. I am looking at it more from the virtualization side.

 

[narzy]

The ISP I used to work at has migrated to HP blade solutions and is very happy with them, personally I tend to like IBM a bit more in terms of support, service and hardware.

What they've done for storage was to use Sun hardware. http://www.sun.com/servers/x64/x4500/index.xml depending on the client and needs these are either mounted at the bottom of each rack or 1 rack of blades 1 rack of storage right next to each other.

That is my 2 cents.

 

[spidey07]

I'll say this - the integrated switches in blade centers are useless turds. Completely featureless turds. Many blade center switches were ripped out and replaced with capable switches with the features necessary in a data center product.

So all that space savings? Not, now you just packed 6u of switches into the same rack and the cable density/mess is insane.

 

[cmetz]

OverWorkedTech, most folks I know are limited by power more than they are by space. Guess which problem blades solve, and guess which problem blades don't solve? So most of the time I've evaluated blade solutions, they end up being a lot more expensive approach that doesn't solve the problems people actually have. Unless the problem you have truly is rack space and only rack space, blade servers are unlikely to be a good solution for you.

Depending on what your application load looks like, would instead look at buying a dual quad-core box with as much RAM as you can stuff it with, and running VMWare ESX. You can get such boxes as 1U if you want to live a little dangerously or really are tight on space, 2U is much more practical though. With infrastructure, you could easily do 24 servers in a cabinet, 192 cores per cabinet, and use VMware to oversubscribe CPUs 2:1. 384VCPUs is a pretty interesting amount of horsepower against 2008 workloads.

 

[tyanni]

Originally posted by: cmetz
OverWorkedTech, most folks I know are limited by power more than they are by space. Guess which problem blades solve, and guess which problem blades don't solve? So most of the time I've evaluated blade solutions, they end up being a lot more expensive approach that doesn't solve the problems people actually have. Unless the problem you have truly is rack space and only rack space, blade servers are unlikely to be a good solution for you.

Depending on what your application load looks like, would instead look at buying a dual quad-core box with as much RAM as you can stuff it with, and running VMWare ESX. You can get such boxes as 1U if you want to live a little dangerously or really are tight on space, 2U is much more practical though. With infrastructure, you could easily do 24 servers in a cabinet, 192 cores per cabinet, and use VMware to oversubscribe CPUs 2:1. 384VCPUs is a pretty interesting amount of horsepower against 2008 workloads.

I have to agree. HP has some nice ENTRY LEVEL rack enclosures which also have a SAN inside, and would have been nice several years ago when we were first looking at ESX and were a very small shop. However, now that we are heavily using ESX, its clear that the best way to go is using several quad-core boxes, such as PowerEdge 2950s or such. We are quite happy w\ 3 PE2950s running 50+ VMs at this point. You can get a 2950 w\ a little local storage and 32GB of ram for around $7000, which makes them quite affordable.

Tim

 

[spikespiegal]

Cmetz hit it right on the head, and this is another one of my pet peeves with data centers.

I think everybody has seen the IBM commercial with the IT geek patting himself on the back for consolidating the entire data center into a pretty rack of blade servers while most of the room is bare. Must be some law of physics I missed in college because the last time I checked servers didn't run better if you shoved them in 2U/4U blade form factors and packed them tight like stereo gear. However, just like gamers obsess about chrome plated water coolers and LED fans data center engineers are obsessed with blade racks and blade enclosures with more built in features than a Lexus. Computing density is something that should be a concern with black holes and women at fat farms - not when critical space is not a factor.

The short form is, unless you are flat out confined for space, avoid blades and just opt for towers. They are generally cheaper and certainly have more configuration options.

If you are planning on a SANs, then plan ahead with the drive configuration for the servers. This will save you money rather than setting up a RAID 5 on each box when you don't need it.

While the older Netburst P4 Xeons are out of production, be aware that they are still being pimped by the server vendors and need to be avoided at all costs.

My preference is for HP right now over Dell or IBM.

VMware ESX - nuff said.

 

[spidey07]

Quick comment on what I'm seeing - 3 very compact midrange unix boxes from the big three (ibm, hp, sun) per rack. Row after row of these, using their own proprietary virtualization technologies. aka, a mainframe.

It's funny how things come full circle. It all really depends on the applications and what you're really trying to accomplish.

 

[drebo]

Take a look at the Intel Modular Server systems.

They're pretty damn slick. Up to 5 disk-less compute modules per rack with integrated SAN and dual GbE. And they're a lot cheaper (as in lower cost) than the HP systems.

I've been very impressed with them.

 

[OverWorkedTech]

Thanks for the replies so far everyone, some good information floating around!
To answer some of the questions that I am picking up on.

1. Space is not an issue for me, I have a large server room with plenty of racks and space available. Power is also not an issue for us.
2. The company I work for was recently acquired, the new owner is pushing to have our servers condensed as possible.
3. I have all of the servers you would expect to find in a large isp providing everything from internet access and webhosting, to telcom services.

Another angle that I am looking at is possibly just moving our webhosting to a blade system. We have a lot of the old Cobalt Raq3,5 servers that are in need of upgrade.
Which brings my to my next question. What is the general consensus on using a virtualized blade system for webhosting?

 

[cmetz]

OverWorkedTech, you've decided on this solution already. I'm sure you'll find some problem to apply it to. Good luck with that approach.