Simple server virtualization options?

[cleverhandle]

I've had a VMWare host running a few virtual servers (for personal web and email, file service, etc.) on a box I built back in 2007. It suffered some kind of hardware failure in the mobo or CPU last week (the day after I left town for the holidays, naturally). Instead of tracking down the fault and rebuilding a 6 year old system, I'm just bringing it back up on a spare box for now. But that box is pretty old too and isn't in a great case for continuous operation, so I'll need to replace it with a more permanent solution in the near future. Then I figure I can take advantage of better cooling cases, lower power components, etc. and maybe update my virtualization platform at the same time.

However, I haven't kept up on server tech in the last several years and don't know what's out there. So for folks in the know, if you were going to build a virtual host to run home hobbyist stuff, what would you recommend for either hardware or software?

The old machine ran the free VMWare server, mainly because it was free and offered nice management options (virtual console, adding/removing/backing up drives, web interface for controlling VMs, etc.). But I'm not attached to it, nor am I unwilling to pay a modest sum for a better solution. The host as well the VMs all run Debian, if that matters. Are there any other solutions I should explore? Again, this is simple home stuff, so ease of management is more valuable to me than blazing performance.

Any recommendations on the hardware side? The old machine lived in an ancient, but sturdy, Antec case that's a little light on the airflow compared to modern offerings. The host's root filesystem as well the VM's themselves lived on a pair of SATA drives in a bootable RAID 1 md array, with my bulk file storage on a separate disk. With the cost of SSD's so much lower now, I would certainly switch the non-bulk storage to that for the sake of power and reliability. Should I still consider a RAID 1 solution with SSD's, or is that just overkill? Any other general hardware recommendations?

Thanks for any pointers you can provide.

 

[mfenn]

For the virtualization software, you can use open-source libvirt with KVM since you're already familiar with Linux. Another option is to use the free VMWare ESXi bare-metal hypervisor, though you need a Windows machine to run the client. If you wanted to spend some money on software, the vCenter Essentials kit is $560 and comes with vCenter server, which provides a web interface.

If it were me, I'd probably start with libvirt so that you can keep on using your md RAID setup rather than needing hardware RAID.

As for hardware, please post the answers to our standard build questionnaire and we can work on a build.

 

[cleverhandle]

Thanks for the reply. I was reading a bit about the VMWare hypervisor and it sounds kind of awkward for my situation. I can probably live with the Windows-only client, but it sounds like backups are something of a bother - and intentionally so, since the free version disables the relevant API used by most backup solutions. My current solution is hacky, but quite efficient for my small setup. Everything backs up onto a big SATA drive connected to the host via ESATA. The system configurations basically never change, so I merely copy an image (with the VM shut down) onto that drive when I do make major changes. Then the home directories (mail from the mail server and docs/music from the file server) get rsync'd into that same drive via a vmnet connection by a weekly cron script. It's hacky and not terribly scaleable, but I can restore all the VM's and data to a new host in a snap if need be. I'm not sure what a comparable solution would look like with a bare-metal hypervisor. I know there are now some backup utilities compatible with the free version out there, but I'm not sure they'll be as straightforward as my current setup. I don't really want to set up a whole backup-across-the-network infrastructure here when my current plug-in-the-drive-weekly setup works well enough.

As for the paid version, $500+ is way too much for my humble purposes. I don't need all those features. I'd pay $100 or $200 tops, but I'm guessing that's way below the radar of commercial solutions.

libvirt sounds promising, though. It looks like the management frontends for it are reasonably well-featured, and I don't need very much anyway. Command-line is fine too, but for routine VM maintenance it's nice to just point-and-click to reboot or get a console. I'll look into libvirt some more as I get closer to building time.

As for hardware, I'm not at the point of a specific build just yet. This project will wait until the summer, since we're going to be doing some major reorganizing of our home, including all the communications and network equipment. I was more just idly curious about what others have done with all the new low-power gear and SSD's in recent years.

Thanks again for the comments.

 

[Chapbass]

Siiiighhhhh, I just typed this big long response and the one time i dont copy my response before hitting submit, the forums eat my post.


Heres the shorter version: Once vmware version 6 comes out, you can manage vmware through the web client completely (the native client wont even be supported) and therefore you wont need a windows box. Version 6 should be out pretty soon i would think.


As far as backups, theres a very easy way to copy vms from the vmware host to your local machine using SCP, and you can do it in a script no problem. It requires that the vm be powered off (which can also be automated), so as long as that isn't an issue then you should be able to do this without issue.


A vm is basically just a collection of files on the host, so its pretty much just like copying files from one linux box to another. I set this up for one of the very remote offices where I currently work. I can send you a link if you're interested.

 

[cleverhandle]

Thanks for the info, Chapbass. And sorry about the post-eating... Maybe I'll give the new VMWare version a run as well, once it's released.

The problem with SCP backups is that we're down to a single desktop running Windows now - we're basically decommissioning our desktops and going to laptops/tablets. And even that one remaining desktop doesn't run 24/7. Is there some kind of small, inexpensive device that I could appropriate as a mini Linux backup server to plug a removable drive into? Otherwise, any kind of SCP-based backup solution is going to add boxes and complexity relative to something with an actual host OS for scripting and device access.

 

[Chapbass]

Just figured out why the forums are eating my posts, its logging me out of the forums every 5 minutes. I blame this new work proxy...piece of junk.

Anyway, I missed in my previous post that you were using vmware right now, I'm guessing you aren't doing something bare metal, using like workstation or vmware player 2.x?

I'm not sure if its possible to backup with a local USB on teh free edition..maybe ill give it a try sometime this week on my home lab. Could also just get a small machine like an intel NUC or maybe even a raspberry pi? Something super cheap and small to sit in the corner.

Kicking off the script is easy as you can schedule it inside of one of the vms to trigger the host...the question is where does the storage sit...

 

[Chapbass]

Hell, just plug it into the host and pass it through to a vm...have a vm be your "backup and scripting" server. like a management box.

http://kb.vmware.com/selfservice/mi...nguage=en_US&cmd=displayKC&externalId=1022290

OR, looks like you can in fact just mount it to the local host:
http://www.virtuallyghetto.com/2012/03/how-to-access-usb-storage-in-esxi-shell.html

looks like he mentions FAT16 though... :\

 

[cleverhandle]

Hell, just plug it into the host and pass it through to a vm...have a vm be your "backup and scripting" server. like a management box.

Hmm... yeah, that could work. I'd want to test out that pass-through capability before deciding, though. The old VMWare Servers (both versions 1 and 2) were really flaky with that.

 

[Chapbass]

Hmm... yeah, that could work. I'd want to test out that pass-through capability before deciding, though. The old VMWare Servers (both versions 1 and 2) were really flaky with that.

I've tested passthrough on esxi 5.1 and 5.5, its EZ-PZ.

 

[cleverhandle]

I've tested passthrough on esxi 5.1 and 5.5, its EZ-PZ.

Good to know, thanks. Did you test USB 2 or 3? I could probably live with 2 if necessary, but 3 would be nice when I rsync a dozen discs worth of new FLAC files.

 

[mfenn]

Heres the shorter version: Once vmware version 6 comes out, you can manage vmware through the web client completely (the native client wont even be supported) and therefore you wont need a windows box. Version 6 should be out pretty soon i would think.

There is a web client in 5.1 and 5.5, but it requires vCenter. Do you think they'll add a web client directly to ESXi in 6? That seems like a pretty big expansion of the ESXi image.

 

[Chapbass]

There is a web client in 5.1 and 5.5, but it requires vCenter. Do you think they'll add a web client directly to ESXi in 6? That seems like a pretty big expansion of the ESXi image.

I think they'll have to. With the native client no longer being supported (or even usable), they need a solution to people without vcenter (even people with paid vSphere but no vCenter). Its already on their forums that they're aware that some features (mainly everything available in 5.1 and 5.5 that doesn't work in the native client) aren't available to those without vcenter...no solution yet tho.

 

[Chapbass]

Good to know, thanks. Did you test USB 2 or 3? I could probably live with 2 if necessary, but 3 would be nice when I rsync a dozen discs worth of new FLAC files.

Ah, I just tested 2, since I was too lazy to go in the back of my server rack when its cold outside (in my garage) .

Looks like 3 is available though:


From VMWare KB: http://kb.vmware.com/selfservice/mi...nguage=en_US&cmd=displayKC&externalId=1022290

There are also now two types of USB controller available for virtual machines to utilize:

EHCI+UHCI controller: This is the classic USB controller allowing the usage of USB 2.0 and USB 1.1 devices.
xHCI controller: This is the newer controller type allowing the usage of faster USB 3.0 devices. Virtual hardware version 8 is required to use this controller.


Heres a quick guide...I did it for my VCAP certification studying a few months ago (not using this guide), but I quick skimmed it and it looks right. http://www.tinkertry.com/usb3passthru/