Building a virtualisation rig

[RedTomato]

Hello AnandTech, my first time posting on here.

I'm being sponsored to study for the MCITP qualification (same as the MSCE for Windows Server 2008) and so I need to build a rig to practice on. I haven't built a PC for 10 years, so I need some advice.

The main use of this is for virtualisation, virtualisation and more virtualisation. It needs to run in VM at the same time a minimum of:

Windows Server 2008, Server 2003, Windows 7 and Vista.

The Server 2008 is the AD domain controller and the 2003 is the Terminal Services host, running a WAN. I also want it to be as hackintosh compatible as possible, and to use reasonably little power, and be reasonably cheap as I have 2 kids to pay for.

Specs so far are:

Chassis: A Powermac B+W chassis that I have lying about and will mod for this. A new chassis is also possible.
Drives: 3x 320GB 7200rpm SATA that I have left over from an old project.
Graphics: none or onboard. Will remote login from my main computer - an Apple macbook.
Monitor: none. If one becomes necessary in future, may get an Dell 2009WA - we have one at work and I like it muchly.

Mobo: up to you.
CPU: up to you.
RAM: a bunch of 2GB sticks, minimum 8GB. I have 4 GB on my macbook which chokes when running Server2008 in a VM. It would be foolish to get a PC with less than twice this.

Not really interested in overclocking. I underclocked an old home server to save power.

I am looking at i7-860 as it is new, cheap, speedy, and low-powered. But the mobos for it only seem to have 4 RAM slots which isn't really enough for hardcore virtualisation. 6 or more would be better.

That seems to indicate the i7-920, but it's more expensive, older, and uses more power. I am also slightly confused by which chips have virtualisation microcode and which don't.

Any advice?

Many thanks

RedTomato

 

[theevilsharpie]

Chassis: What you have
Drives: 3x 320GB SATA in RAID 0
Graphics: Remoting into a VM console is slow because RDP can't optimize a VM console as it can the host desktop. A cheap low power graphics card will suffice. Consider getting a dual-monitor setup.
Mobo: ASUS P6T SE
CPU: Intel Core i7 920
RAM: As much DDR3 1333 as you can cram in
Software: Windows 7 Pro / VMware Workstation 7

Don't worry too much about Core i7's power consumption. Unless you're virtualizing something CPU intensive, it'll stay in its lower power states most of the time.

As for virtualization features, the big one in the Intel world is Intel VT, which lets you run 64-bit virtual machines. Other virtualization assist features are designed primarily to improve performance, and would only be applicable to a bare metal virtualization environment.

 

[Thor86]

I would rather get more disks and RAID 1+0 the disks to get better I/O. I'm using on board ICH9R on a P35 chipset, and I/Os are great.

If you do not need heavy cpu, that I7 or I5 are overkill. Any cheap Q6600 even will do nicely for up to 8 + VMs for 8GB of memory.

 

[theevilsharpie]

I would rather get more disks and RAID 1+0 the disks to get better I/O. I'm using on board ICH9R on a P35 chipset, and I/Os are great.

If you do not need heavy cpu, that I7 or I5 are overkill. Any cheap Q6600 even will do nicely for up to 8 + VMs for 8GB of memory.

The biggest bottleneck for a little virtual lab will be memory, and the Core i7 is worth it in higher memory capacity alone.

More spindles would help as well, but a single high-performance SSD would be faster than a bunch of HDDs in a RAID 10.

 

[Mir96TA]

I would suggest Get as much more memory you can,
Even if you get a slower memory but as long it is more, it is better.
Do not bother with SSD. Just use that money to get more Spindles.
Set Raid 0, install Arcnoics (sp) Back the Stuff to 8 GB DvD (OS Stuff) you may need 2-3 depend what you have.
F Round with OS Bring it down etc etc......... And when you want you can restore it.

 

[RedTomato]

OK, I've done some pricing up.

After looking at the hackintosh forums e.g. insanelymac.com, the cannonical i7 rig seems to be this:

i7-920 £205
Gigabyte GA-EX58-UD5 £182
Branded memory 12GB £200 (6x2GB) (OCZ Obsidan)

Total: £587 (excluding case etc)(Prices are from overclockers.co.uk, and are broadly similar across other sites)

At insanelymac.com, the general feeling seems to be that i7-860 + p55 is not yet fully newbie friendly yet. Several people are still working on unposted guides which should be posted in a few weeks, as the new i7-860 iMacs have not quite arrived yet.

Temporary i7-860 parts list:

i7-860 £227
Gigabyte GA-P55-UD4P £136 (recomended in Ars Technica's latest HotRod guide, hackintosh compatibility is currently unknown)
Branded memory 8GB £126 (4x2GB)(OCZ Gold)

total: £489

A fair bit cheaper, but with the disadvantage of less RAM, and it seems more complicated at the moment to install osx on a p55 board.

Will probably wait for the iMac i7-860 teardowns / hackintosh guides to start arriving.

PS Where is AnandTech's latest System Buyer's Guide? There's been no update since July, and before that there were monthly updates. Has Anand decided to stop posting them?

 

[theevilsharpie]

I don't think it's a good idea to build such a large purchase around an operating system that it can't legally run and whose vendor is taking steps to disable the OS on unauthorized hardware.

 

[Keitero]

I agree with sharpie on that note. Tread lightly on the hackintosh scene. It may work now, but sooner rather than later, you'll bound to have headaches.

As for the machine, I'd say a Core i7 920 + Asus P6T SE or Gigabyte EX58-UD4P/UD5 or any SuperMicro 1366 motherboard (most come with onboard video). + 16GB of DDR3-1333. Will serve you just fine. Run Windows 2008 Server x64 or 2008 Server R2 (if you are studying for that) and use Hyper-V for your Virtual machines.

 

[theevilsharpie]

VMware Workstation is much more suitable for creating the type of VM labs that the OP is looking for. Hyper-V is faster, but has no ability to create teams or linked clones (that I'm aware of), nor can it overcommit memory.

It would be useful to reserve a portion of disk space to be able to practice with Hyper-V, since I'm sure it's part of the MCITP Server/Enterprise Admin exams, but I wouldn't use it for general study.

 

[Keitero]

That is true, but I'm sure that the test will consist of topics that relate to Hyper-V.

 

[RedTomato]

VMware Workstation is much more suitable for creating the type of VM labs that the OP is looking for. Hyper-V is faster, but has no ability to create teams or linked clones (that I'm aware of), nor can it overcommit memory.

It would be useful to reserve a portion of disk space to be able to practice with Hyper-V, since I'm sure it's part of the MCITP Server/Enterprise Admin exams, but I wouldn't use it for general study.

Yes you're right, I'm aiming for the Enterprise Admin exams. Just ordered the books for it, and hopefully they will arrive next week. Have decided to do the new Windows 7 prerequisite for my first module. I just set up a Windows 7 PC for work last week, and so far, my impression is 'like Vista but slightly more grown up'.

Thanks for the tips about VMWare Workstation and Hyper-V - I will bear them both in mind.

Someone else suggested a Gigabyte P55M mobo. I'm quite intrigued, as it's lovely and small. If I could cope with the 8GB of RAM for my virtual machines, I could easily carry a micro-ATX chassis to work as needed. Cheapness is a bonus too.

 

[theevilsharpie]

For virtualizing Vista/7/Server 2008, 8GB is a little low. With Vista/7, the main OS will use about 1.5GB (about 750MB with Hyper-V on Server 2008 Core), which will only leave about 6GB for VMs. You might be able to get 4-5 VMs out of that, which isn't enough for some of the larger labs.

Your best bet for a VM lab would be:
- Core i7 with 12GB of RAM
- Windows 7 Pro/Ent/Ult 64-bit
- As many ReadyBoost drives as you can stuff into the USB slots
- SSD to hold the page file and VMs

 

[Chiropteran]

I think 8GB is plenty. I'm really surprised your VM ran poorly with 4GB on your laptop, are you sure it was memory and not lack of CPU or something else? You should be able to run a windows 2008 server vm on 2GB easily and so 8GB would give you enough to run all 4 VMs. Vista and Windows 7 run fine with 1.5GB, so you could even bump server 2008 up to 3GB if you felt you needed more ram. If you use ESXi as your host, the memory impact is basically nonexistent.

I actually just bought a machine for similar purposes, but I took the lazy route and bought a server from Dell, the new t110s are fairly cheap ($400~) for a i7 based system, although the ddr3 memory is expensive to upgrade.

 

[RedTomato]

Thanks for your contributions. Two posts, two opposing opinions.

My MacBook with 4GB runs 2k8 more or less fine in VM, but when I try to run it with Vista at the same time that's when it chokes.

As the evilsharpie pointed out, I need to be running 2k8, 2k3, 7, vista all at the same time to study their interaction. That's for my exams, and also to replicate my workplace IT setup. We're currently using 2k8 as a domain controller and my boss has asked me to set up a WAN with a 2k3 server as a Terminal Services host.

I looked at the Dell T110s. In the US, ~$400, yes. In the UK, over £650 (inc 15% tax) for a base config with 1GB RAM. With shipping, nearer £700.

Dell UK doesn't seem very keen on selling them. Their UK page for the T110 has images, memory options, and text for the T605 which is an AMD dual CPU beast. Oh dear.

http://premierconfigure.euro.dell.com/dellstore/config.aspx?oc=PET110

I'm getting more attracted to a micro-ATX P55m unit, as I live in a small London flat and it would be good to have something portable.

ESXi looks interesting - thanks for the heads up. I didn't know about it - like I said, I'm still learning all this. I was figuring on allowing ~2GB for the host OS, but with ESXi seems I can use that for one of the VMs. 4 VMs should be enough for me.

 

[waffleironhead]

I'm studying for the 70-640 exam and all of the labs I do require one w2k8 vm, and a vista vm, plus a few doing stuff in core. All of the labs were/are done on a x4 620 with 4 gigs of ram at home. I will note that the school rooms have pentium e2180's with 8 gigs of ram and they run all of the labs as well.

FOr most of the stuff you will be doing the vm's only require 512 ram allocation as you are just practicing configuration, not actually putting it into production.

Why do you think you need so much horsepower/ram?

 

[theevilsharpie]

I'm studying for the 70-640 exam and all of the labs I do require one w2k8 vm, and a vista vm, plus a few doing stuff in core. All of the labs were/are done on a x4 620 with 4 gigs of ram at home. I will note that the school rooms have pentium e2180's with 8 gigs of ram and they run all of the labs as well.

FOr most of the stuff you will be doing the vm's only require 512 ram allocation as you are just practicing configuration, not actually putting it into production.

Why do you think you need so much horsepower/ram?

If you're only using two VM's for the labs, you're doing the absolute bare minimum, which won't give you the experience needed to pass the exam. This is especially true when studying infrastructure, clustering, or anything that involves multiple sites linked with a slow connection. While studying the MCSE 2003 exams, it wasn't unusual for me to have a 12-15 VM lab up and running, and I doubt the MCITP exams are much different in that regard.

Also, you can skimp on the RAM, but that just means the VM hits the page file. If it's running on an SSD, the performance isn't going to immediately crater, but it'll still be slow enough to be distracting.

 

[waffleironhead]

If you're only using two VM's for the labs, you're doing the absolute bare minimum, which won't give you the experience needed to pass the exam. This is especially true when studying infrastructure, clustering, or anything that involves multiple sites linked with a slow connection. While studying the MCSE 2003 exams, it wasn't unusual for me to have a 12-15 VM lab up and running, and I doubt the MCITP exams are much different in that regard.

Also, you can skimp on the RAM, but that just means the VM hits the page file. If it's running on an SSD, the performance isn't going to immediately crater, but it'll still be slow enough to be distracting.

Good point about the clustering. I guess my kneejerk reaction was that there was no reason that 'most' of the labs couldnt be done on his macbook or a low end workstation.

I guess the system needs to be built around the most intensive thing that needs to be done.

 

[RedTomato]

If you're only using two VM's for the labs, you're doing the absolute bare minimum, which won't give you the experience needed to pass the exam. This is especially true when studying infrastructure, clustering, or anything that involves multiple sites linked with a slow connection. While studying the MCSE 2003 exams, it wasn't unusual for me to have a 12-15 VM lab up and running, and I doubt the MCITP exams are much different in that regard.

Thanks for passing on that real-world experience. I can't imagine needing 12-15 VMs on the go. What was in them?

I've ordered these books http://www.amazon.co.uk/Self-Paced-T.../dp/0735625727 (MCITP Self-Paced Training Kit [Exams 70-640, 70-642, 70-643, 70-647]: Windows Server 2008 Enterprise Administrator Core Requirements Book/CD/DVD) so when they arrive in a week or two I'll have to do a quick scan and see what's the biggest VM lab they need.

 

[theevilsharpie]

Thanks for passing on that real-world experience. I can't imagine needing 12-15 VMs on the go. What was in them?

At the minimum, I'd have at least a firewall running NAT (so I can run services like DHCP without conflicting with my local network), a domain controller, a client, and an administrative workstation from which I'd perform my work.

For a more advanced lab (in this case, a cluster lab), I'd have a firewall, two domain controller, three windows cluster servers, three NLB cluster servers, an iSCSI SAN server, a client, and an administrative workstation.

 

[classy]

I work with virtualization every single day. For this and by the way I am a MCSE , if I am building from scratch to accomplish what you want to do, on the software side I would probably do Microsoft Virtual PC or Virtual Server with a cheap AMD quad core with plenty of cheap memory. Get good disks, preferably the fastest serial drives you can get or afford. For study or development you don't need an expensive cpu, because most of your perfomance feel will come from the memory and the disks. If you were doing some work related stuff like a development domain or running production servers then you need some cpu power. But to be able to run 4-6 vms which is all you need you'll be fine. AMD processors are excellent for virtualization, especially in this kind of scenario. Nothing wrong with I7 either, but your money could better spent else where. I wouldn't pay for Virtualization software like VMWare workstation, the Microsoft free stuff and ESXi are better actually. So just save that cash expense. The best thing would be to find a cheap rack server like a Dell 2850 and run ESXi. While you won't be able to run 64 bit guest vms only 32 bit, one of those with ESXi will provide plenty of vm practice goodness. I work in VMWare at work almost everyday. Far from an expert, but I got some good experience under my belt.

 

[theevilsharpie]

I wouldn't pay for Virtualization software like VMWare workstation, the Microsoft free stuff and ESXi are better actually. So just save that cash expense.

No way. I wouldn't be able to run a lot of my labs (particularly those involving multi-site) with ESXi or any of Microsoft's stuff, nor would I be able to set up labs as quickly. VMware Workstation was well worth the investment.

Your answer would be fine if his focus of study was virtualization, but in this scenario it's used more as a tool for rapid lab work. Also, not being able to run 64-bit VMs is a big problem if his study involved Server 2008 R2.

I also disagree with you on the platform. For lab work, disks are only hit when spinning off VMs or when the VMs are paging. RAM is far more important.

 

[classy]

No way. I wouldn't be able to run a lot of my labs (particularly those involving multi-site) with ESXi or any of Microsoft's stuff, nor would I be able to set up labs as quickly. VMware Workstation was well worth the investment.

Your answer would be fine if his focus of study was virtualization, but in this scenario it's used more as a tool for rapid lab work. Also, not being able to run 64-bit VMs is a big problem if his study involved Server 2008 R2.

I also disagree with you on the platform. For lab work, disks are only hit when spinning off VMs or when the VMs are paging. RAM is far more important.

I don't really want to get into an argument but you are flapping a bunch of junk. First off when studying, he will at most have two to 3 vms going. He will spend most of his time on one VM, one. He will use two or 3 when testing replication and the setup of some domain services. ONLC one the top tech teaching sites in tri-state area uses Virtual PC as its VM software. Obviously you haven't looked at one iota of virtual performance, because in a workstation VM type scenario with 2-3 vms, a slow or one disk can slow down your VMs big time, especially using any of the Virtulaization software that runs on top of another OS. Which is why I suggested using ESXi. And you can easily setup switches to emulate different sites in any of them. They just won't be able to have internet connections, so I don't know what "labs" your talking about. I also used Virtaul Server from Microsoft for home study when I got my MCSE 3 years ago. I do this stuff for a living, I actually have a fulltime job where I worked with virtualization. I have worked with VMWorksation for over 5 years now, VMWare Infrastructure for 2 1/2, and Microsft virtual stuff for over 4 years. For a practice setup he doesn't need to spend a lot of money which is what many are suggesting. Thats all I trying to help him avoid. And one more thing, there is zero difference in the setup or use of any 64 bit server over 32 bit. None, zip, nada. But when you setup a box with 24 gigs of ram you'll need a 64 bit OS. He can study night and day on 32 bit and never know the difference.

 

[Zstream]

No way. I wouldn't be able to run a lot of my labs (particularly those involving multi-site) with ESXi or any of Microsoft's stuff, nor would I be able to set up labs as quickly. VMware Workstation was well worth the investment.

Your answer would be fine if his focus of study was virtualization, but in this scenario it's used more as a tool for rapid lab work. Also, not being able to run 64-bit VMs is a big problem if his study involved Server 2008 R2.

I also disagree with you on the platform. For lab work, disks are only hit when spinning off VMs or when the VMs are paging. RAM is far more important.

I would love to know how you are running all of those VM's with VMWorkstation. ESXi or Hyper-V is by far the best bare metal virtualization solution. Trying to run more then two VM's under Virtual PC or VMWorkstation is rather difficult.

Pretty much a novice when it comes to virtualization but still recognize a bad setup when I see it.

BTW my setup:
i860
8gb ram
Raid-0 - Two spindles

The biggest problem was that I was I/O bound (hard drive) with more then two running at the same time. If I rebooted both of them it took forrrrrrrrever to load.

 

[theevilsharpie]

I don't really want to get into an argument but you are flapping a bunch of junk. First off when studying, he will at most have two to 3 vms going. He will spend most of his time on one VM, one. He will use two or 3 when testing replication and the setup of some domain services. ONLC one the top tech teaching sites in tri-state area uses Virtual PC as its VM software. Obviously you haven't looked at one iota of virtual performance, because in a workstation VM type scenario with 2-3 vms, a slow or one disk can slow down your VMs big time, especially using any of the Virtulaization software that runs on top of another OS. Which is why I suggested using ESXi.

I guess my computer must have missed that memo, because my labs often have about a dozen VMs or so, and my computer is about five years old.

No, VMware Workstation isn't going to be lightning fast, but it doesn't need to be in a lab environment. ESXi and Hyper-V will have better raw performance, but won't have nearly the flexibility of VMware Workstation. The only product that can match it for lab work is VMware Lab Manager, and that's probably outside of the OP's budget

And you can easily setup switches to emulate
different sites in any of them.

VMware Workstation allows you to configure networks with specific amounts of bandwidth, so you can simulate slow connections like T1s, or connections with packet loss like wireless. Neither ESXi nor any of Microsoft's products can do this. Some of the Microsoft exam questions (particularly those involving Active Directory or DFS) will ask about behavior over slow links, which can't be replicated over a fast network.

They just won't be able to have internet connections, so I don't know what "labs" your talking about. I also used Virtaul Server from Microsoft for home study when I got my MCSE 3 years ago. I do this stuff for a living, I actually have a fulltime job where I worked with virtualization. I have worked with VMWorksation for over 5 years now, VMWare Infrastructure for 2 1/2, and Microsft virtual stuff for over 4 years.

That's nice, but you clearly have little experience with VMware Workstation or you wouldn't be recommending products like ESXi or Virtual Server in it's place. They aren't well suited for the rapid building/reconfiguration/teardown that VMware Workstation is designed to support.

For a practice setup he doesn't need to spend a lot of money which is what many are suggesting. Thats all I trying to help him avoid. And one more thing, there is zero difference in the setup or use of any 64 bit server over 32 bit. None, zip, nada. But when you setup a box with 24 gigs of ram you'll need a 64 bit OS. He can study night and day on 32 bit and never know the difference.

Server 2008 R2 is only available in 64-bit, so not being able to run a 64-bit VM will put a quick stop to that course of study.

 

[theevilsharpie]

I would love to know how you are running all of those VM's with VMWorkstation. ESXi or Hyper-V is by far the best bare metal virtualization solution. Trying to run more then two VM's under Virtual PC or VMWorkstation is rather difficult.

I don't know where this idea that host-based virtualization can't handle more than a handful of VM's comes from. It's not difficult at all. Linked clones barely take up any disk space, and in a lab environment, the CPU usage is practically nil. My only real bottleneck was memory utilization. Even my laptop can run 3-4 VMs without breaking a sweat.

Pretty much a novice when it comes to virtualization but still recognize a bad setup when I see it.

Whether or not it's a bad setup depends on how it's used. ESXi is ideal for production use where performance and reliability is critical. Workstation is better for testing or any other use that requires flexibility in how the VMs are connected.

The biggest problem was that I was I/O bound (hard drive) with more then two running at the same time. If I rebooted both of them it took forrrrrrrrever to load.

In Workstation, you can set the order in which VMs load, as well as a delay. I had my VM's boot up in 30-second intervals, which kept the I/O load reasonable.