How many virtual machines can I run?

[jaydee]

Hi all,

I'm new to VM's, but I've been tasked to set up a bunch to hammer on a server. I will be using VMware Player. I have two different physical hardware setups:

Core i7-3770 (4 cores, 8 threads) @ 3.4 GHz base
4GB of DDR3 1600, 500GB HD @ 7200 RPM
Win 7 Pro License

E5-2620 (6 cores, 12 threads) @ 2.0 GHz base
8GB of DDR3 1600, 500GB HD @ 7200RPM
No OS license, VMware to be loaded on some form of Linux (open to suggestions, only hands-on experience is with Ubuntu derivatives and Mint)

The hardware I am trying to replicate, is as many of the following as possible: Intel Atom E660 (1.3GHz, 512kB L2), with 2GB of RAM, 8GB Solid State HD, running Debian 6.0.7 no DE.

My specific question is, what is a reasonable amount of VM's I can expect to run on each of the above hardware setups? If I were to double or quadruple the RAM on each of the above hardware setups, how does that effect the number of VM's I can run?

Thanks!
jaydee

 

[Ketchup]

You want more RAM.

With your first setup, you will only get 1 VM if you want the 2 GB of dedicated memory for it. And if you do that, your host machine will run like crap.

For the second, you should be fine with 2.

 

[Snoop]

Ram and disk subsystem are the key. Grab some memory and an SSD to put the VM's on. A box with VM's running on an SSD, and 16GBs of ram should be able to run 6 maybe 7.

It also may depend on what exactly the VM's are doing. My above estimate is assuming you are just trying to run the VM's. If you are doing something cpu intensive, my estimate would likely change.

 

[Berryracer]

I wouldn't even run 1 VM with 4 GB RAM

I usually allocate 4 GB for each VM

 

[Ketchup]

I wouldn't even run 1 VM with 4 GB RAM

I usually allocate 4 GB for each VM

Like Snoop said, it depends on what you are using the VM for. Most days I run two VMs with XP. One has 512 MB of RAM, the other has 1 GB. The 512 one runs fine, but the other ran like crap when it only had 512 MB.

 

[Berryracer]

Like Snoop said, it depends on what you are using the VM for. Most days I run two VMs with XP. One has 512 MB of RAM, the other has 1 GB. The 512 one runs fine, but the other ran like crap when it only had 512 MB.

yeah you're right. They would run. But for my liking, and with the prices of RAM these days, it is really worth it to add like 8 GB more to his system

 

[Ketchup]

yeah you're right. They would run. But for my liking, and with the prices of RAM these days, it is really worth it to add like 8 GB more to his system

For what he seems to want, I totally agree.

 

[Gillbot]

If I try to run more than 4-5 on my laptop I have issues. The SSD helped a lot and it has 16gb of ram, but the more I do in the VM's the more the i5 struggles to keep up.

My "server" does better but it really needs upgraded to an SSD to host the VM's. It's a 3770k with 32gb of ram and it can run a few without skipping a beat, but the basic "green" HDD really holds it back and things become sluggish as I/O gets bound.

My main rig handles them without a single issue. 3930k, 64gb of ram and SSD.

You can see, as ram and I/O speeds increase, the more and better the VM's run.

 

[lxskllr]

There's tools for stress testing servers without using actual machines. If you need actual machines, Maybe a minimal GNU/Linux distro would work for your purposes. Something like TinyCore uses minimal resources, and you could probably use 10-20 times more VMs than you could with Windows.

http://tinycorelinux.net/

Edit:
Or maybe disregard everything I said :^D I'm not entirely clear on the goal.

 

[dave_the_nerd]

There's tools for stress testing servers without using actual machines. If you need actual machines, Maybe a minimal GNU/Linux distro would work for your purposes. Something like TinyCore uses minimal resources, and you could probably use 10-20 times more VMs than you could with Windows.

http://tinycorelinux.net/

Edit:
Or maybe disregard everything I said :^D I'm not entirely clear on the goal.

Yeah, we need more info from OP.

That said, yes, use the hex core rig, add another 8GB of RAM if possible. Store VMs on a SSD or dedicate a separate HDD to each. And a separate NIC. Otherwise, your VMs won't really be able to even run fast enough to stress the server properly.

You'll probably want something similar in design philosophy (if not to the same extreme) as the Anandtech NAS testbed.

 

[jaydee]

Thanks for all the info guys. I've requested another 16 GB's for the first system, and 16 GB's for the 2nd (which I have two of) for a total of 20 and 24 GB respectively. I'll see what I can do for SSD's. The 2nd configuration is actually a 1U server with SAS bays, I haven't had much luck finding 2.5" SATA to SAS converters yet, but I assume they're out there somewhere. I can deal with a consumer drive, this is for testing purposes, not for service mode.

 

[Ketchup]

Ideally, a separate hard drive for each VM is ideal. Practically though, not so much. I did make a separate partition for each VM on mine, and the overall performance of the machines and the host improved quite a bit. But it is still one drive, so whenever I use more than one machine at a time, the hard drive starts growling.

 

[jaydee]

As for the actual function of the system and roles of the VM's, they are essentially serving the purpose of routers. We are testing this type of system, running on a closed gigabit network.

1. Xeon Servers (3-5)
2. Atom machines (up to 200 per server)
3. ARM-based nodes (up to 30 per Atom machine)

None of the above generates it's own data, they are only a communication system. The information is generated upstream of the Xeon servers, and downstream of the ARM nodes. We don't have 200 Atom machines on site, so I am trying to make as many VM's as I can (let's say 40) to emulate them, then look at the CPU, RAM usage, disk I/O of the Xeon server, multiply it by 5 to see if the system can handle the 200 Atoms per server requirement.

The Atom-based machines run a web server (which displays information in real-time), record log files, send SNMP's, maintains connections with up to 6 other Atom's, one server & ARM nodes via heartbeats & polling, and maintains a mix of static & dynamic routing tables.

My hope is, I can run about 10 VM's on the i7-3770 system, and maybe 15 on each E5-2620 system. That would give me 40 VM's total.

Edit: I should be able to grab a 120GB SSD for the i7. I'm not sure how to increase I/O performance of the E5's, as I'm not sure of SATA and SAS are pin-for-pin compatible and I can't find SAS SSD's for anywhere near a reasonable price.

I can temporarily borrow 6 more 7200RPM SAS drives, for 4 drives each. Would I be ok running 5 VM's per 7200RPM 500GB SAS drive?

 

[deaner]

Its quite easy:

Go buy a server with 2 X Xeon Quad's with hyperthreading (or AMD chips) and 72Gb or more of Ram. Dedicate it with a Type 1 hypervisior of your choice, XenServer, Xen, Esxi or Type 2 - MS Hyper-V. Happily run a ton a VM's - as i just did.

 

[ultimatebob]

Its quite easy:

Go buy a server with 2 X Xeon Quad's with hyperthreading (or AMD chips) and 72Gb or more of Ram. Dedicate it with a Type 1 hypervisior of your choice, XenServer, Xen, Esxi or Type 2 - MS Hyper-V. Happily run a ton a VM's - as i just did.

Yep, it's dead simple to fix if you have $10,000 for the server that you just mentioned

 

[ultimatebob]

As for the actual function of the system and roles of the VM's, they are essentially serving the purpose of routers. We are testing this type of system, running on a closed gigabit network.

1. Xeon Servers (3-5)
2. Atom machines (up to 200 per server)
3. ARM-based nodes (up to 30 per Atom machine)

None of the above generates it's own data, they are only a communication system. The information is generated upstream of the Xeon servers, and downstream of the ARM nodes. We don't have 200 Atom machines on site, so I am trying to make as many VM's as I can (let's say 40) to emulate them, then look at the CPU, RAM usage, disk I/O of the Xeon server, multiply it by 5 to see if the system can handle the 200 Atoms per server requirement.

The Atom-based machines run a web server (which displays information in real-time), record log files, send SNMP's, maintains connections with up to 6 other Atom's, one server & ARM nodes via heartbeats & polling, and maintains a mix of static & dynamic routing tables.

My hope is, I can run about 10 VM's on the i7-3770 system, and maybe 15 on each E5-2620 system. That would give me 40 VM's total.

Edit: I should be able to grab a 120GB SSD for the i7. I'm not sure how to increase I/O performance of the E5's, as I'm not sure of SATA and SAS are pin-for-pin compatible and I can't find SAS SSD's for anywhere near a reasonable price.

I can temporarily borrow 6 more 7200RPM SAS drives, for 4 drives each. Would I be ok running 5 VM's per 7200RPM 500GB SAS drive?

That's probably wishful thinking. Even with the extra memory, I doubt that you could run more than 7 or 8 VM's on the i7-3770 box before the system becomes unbearably sluggish.

 

[jaydee]

That's probably wishful thinking. Even with the extra memory, I doubt that you could run more than 7 or 8 VM's on the i7-3770 box before the system becomes unbearably sluggish.

Thank you for the input. Out of curiosity, what will be my limiting factor? And what about 15 VM's on the Xeon's?

 

[Snoop]

I can temporarily borrow 6 more 7200RPM SAS drives, for 4 drives each. Would I be ok running 5 VM's per 7200RPM 500GB SAS drive?

That may be a bit aggressive though it is really difficult to estimate without further info. In my experience, SSD drives are just not comparable to mechanical drives when it comes to running multiple Virtual machines. With a mechanical drive, if you do something with any I/O intensity and multiple VM's fighting for the I/O time, the drive will grind like crazy. SSD seem to thrive in this same environment.

I believe SAS controllers can handle SATA drives (From Wikipedia) though I have never tried this:
SAS offers backward compatibility with second-generation SATA drives. SATA 3 or 6 Gbit/s drives may be connected to SAS backplanes, but SAS drives cannot connect to SATA backplanes.[1]

 

[ultimatebob]

Thank you for the input. Out of curiosity, what will be my limiting factor? And what about 15 VM's on the Xeon's?

It's hard to tell without trying it, but I'm thinking that disk I/O is going to become a factor around VM #6 or #7. It depends on how busy the VM's get when writing to their log files.

The Xeon server will have the same problem unless you give it more drives to work with. Either an SSD or more drives would help in both systems.

 

[Ketchup]

It's hard to tell without trying it, but I'm thinking that disk I/O is going to become a factor around VM #6 or #7. It depends on how busy the VM's get when writing to their log files.

The Xeon server will have the same problem unless you give it more drives to work with. Either an SSD or more drives would help in both systems.

Just to add to this, SATA III and a separate disc drive/VM will give you best speed and lifespan for multiple VM's IMO. At that point you will only have to worry with the limitation of the bus, which would be 600 MB/s. Like ultimatebob mentioned, 6 of those would give you 100 MB/s per drive, which would still be plenty (about as much as the original SATA standard, which a parallel drive still wouldn't touch).

 

[duncan-idaho]

That's probably wishful thinking. Even with the extra memory, I doubt that you could run more than 7 or 8 VM's on the i7-3770 box before the system becomes unbearably sluggish.

Absolutely not more than seven as you need 1 thread per VM and that includes the host system.

Hosting a bunch of VMs is a good example of why the 2011 platform exists. I see gamers with 2011 chips all the time and just shake my head at the ignorance and waste, but for building virtual labs it sure is nice having 8 core 16 thread per socket ^_^

 

[ultimatebob]

Oh, and shame on me for not mentioning this earlier... On the Xeon server, try running a bare metal hypervisor like VMWare ESXi instead of VMWare Player. You should get better performance due to less memory usage from the host OS and the various virtualization optimizations that VMWare puts in their kernel.

It probably will not have the right drivers for your desktop PC, but the server should work.

 

[smakme7757]

RAM will be your biggest limiting factor at the moment. After that it will be your storage.

You should look into getting 2x 500GB Samsung EVO SSDs, one for each system if you need to run a few machines on both. They are very nicely priced. Then with such speedy storage you should be able to create as many VMs as your RAM will allow.

I'm currently running Hyper-V on a Xeon (4/8) with 32GB RAM.
2x 2TB in RAID 1
4x 1TB in RAID 10
1x 500GB Samsung EVO as test LAB (On the way)

I've got my file server VM running on the RAID 1 and my webserver and a few other VMs on the RAID10. Currently I'm using the RAID 10 for my lab as well which is putting strain on the array, so I've ordered one of those 500GB Samsung SSD to act as my personal LAB on the server.

If your space requirements aren't massive then you should definitely look at the Samsung EVO or the Crucial M500 for cheap SSD storage. I would have gotten the 960GB model from Crucial, but i just don't need that much space.

Having such fast storage and high IOPS should allow you do maximize your RAM usage before you run into a massive storage bottleneck.

Edit: The Samsung EVO is rated for the following IOPS:

• Random READ up to 98,000 IOPS
• Random WRITE: up to 90,000 IOPS

Which is really nice for the price.