Need suggestions for a virtualization/media server

[Arkitech]

I recently started a new job where I will be training to take on a lot of different roles (server administration, network architecture, cloud implementation, linux admin). I would like to build a server I can run a number of VMs concurrently to assist me in my training. Just off the top of my head, I would say at any given moment I may want to run maybe 3-5 VMs at once. There won't be any heavy loads since these are just VMs for testing, the exception would be the media serving component.

I inherited a Precison T7400 from my last job, but I can't get any video so I really don't know the exact state of the hardware yet. I need to test the hardware to see what's usable. But I would also be willing to spend maybe $500-1000 to build something from scratch or buy an existing box if need be.

Thanks for any information and I would really be appreciative of a source where I can research on how to plan out a server build.

 

[dave_the_nerd]

There are a pretty wide variety of used servers on eBay. (Lenovo i3-base systems with ECC RAM are pretty common.) I'd go for a pedestal style rather than a rackmount if it's going to be in your house. (They're usually a lot quieter.)

The "server" in my signature is a pretty decent VM host. Supermicro workstation boards are a little pricey, and they don't support as much RAM as a dedicated server (only 32GB max in this one.) But it has IPMI, dual NICs, and all the other server goodies, and the low-end Pentium/Celeron dual cores are sufficient for lab/learning rigs. I shopped sales and bought some parts open-box returns, so it only cost me around $550 plus the cost of HDDs.

For more resources, reviews, build examples, etc., you might check out http://www.servethehome.com/

 

[XavierMace]

I'll second the Supermicro recommendation if you're going to be building your own. The RAM restriction does vary depending on board. Both of my Supermicro boards support 192Gb of RAM.

I'd also take a look at the guide I wrote a while back: http://forums.anandtech.com/showthread.php?t=2405727. Servethehome (Dave's link) is also a great resource.

 

[zir_blazer]

I can't believe that dave_the_nerd is blaming the Supermicro for "only 32 GB" in his Motherboard, since you can't go above that in that platform. RAM support depends on platform and number of slots.

On old LGA 1150 Haswell platform, with 4 DDR3 slots you max at 32 GB, period, since the biggest Unbuffered modules are 8 GB, and you can't use more than 4 modules. You will NEVER see more than that there.
For Skylake LGA 1151 you can use up to 4 Unbuffered 16 GB DDR4 modules, so currently you max out at 64 GB. I don't know if there will be Unbuffered 32 GB modules, and in case that they will exist, the current Skylake Integrated Memory Controller needs to have forward compatibility support for higher density DRAM ICs to use them.
Both LGA 1150 and 1151 can also be used with Unbuffered+ECC modules of their correct type (DDR3 or DDR4) if you also use a Xeon E3 (Or some low end Pentium/Core i3s models) and a Motherboard with a C Series Chipset.

LGA 2011-3 Haswell-E has 8 DDR4 slots instead of 4, so you can use 16 * 8 = 128 GB. And besides adding in Unbuffered+ECC support, Xeons E5 can use Buffered (Registered) DDR4 modules, which allows for both modules of higher capacity and more slots (I saw up to 12) per Processor.
Supermicro does have Workstation Motherboards for LGA 2011-3, where you can spend a truckload of money in RAM if you want to. Note that Workstation Motherboards usually have integrated Audio, Servers models do not, plus usually they always have IPMI instead. The X10SLM-F (IPMI, no Audio) is Server, an X10SAT (No IPMI, Audio) is Workstation.



With 1000 U$D budget I suppose you can fit a Haswell/Skylake Xeon E3, a Supermicro Motherboard, and 32 GB ECC RAM. Also remember to check for Intel iGVT-g/XenGT/KVMGT, currently supported in Haswells, but next release would support Skylakes, too. You will love it.

 

[mvbighead]

From a whitebox standpoint, the hardware 'should' work:
http://www.vm-help.com/forum/viewtopic.php?f=13&t=2484

Thing is, this individual tested with ESXi 4.1. I would recommend try to get your hardware to work prior to investing in ANYTHING additional. You may be able to load ESXi 6.1, you may not. If you can, then you may want to invest in additional resources...

From what I have read on the T7400, it could work. You may need RAM, and it appears this would work:

http://www.amazon.com/Memory-Precis...449504745&sr=8-1&keywords=precision+t7400+ram

The above is very cheap for 16GB, but I would suspect it may be one of your best bets.

I'd recommend throwing an SSD or two in there if you don't use RAID. As disk is shared, spinning SATA disks are going to be slow (and can be unbearably so at times). If you do use RAID, you could potentially do ok with a RAID10 on some spinning disk, especially if you want capacity (file server).

As for the recommendations above... they're good if you're trying to build a production home box of sorts, but if you're just figuring out the nuts and bolts, you don't need to over complicate it. Just make sure the MB has been tested and validated by someone. And honestly, for most builds, just about anything works. You simply need:

A supported Gigabit NIC (Intel are readily available: http://www.amazon.com/Intel-Gigabit...&qid=1449505366&sr=8-1&keywords=intel+gigabit). You can download MIBs and inject them into a disk or add them to your host to support other NICs (Realtek for instance)

A supported disk. Standard disk controllers work, or real RAID controllers (this could work: http://www.amazon.com/HP-462919-001...d=1449505468&sr=1-12&keywords=raid+controller). You do not typically want to use a consumer grade type of RAID controller that is built into the board as they are most likely not supported. As to disk, SSD is your best best for performance. Just need to be aware of how big of VMs you plan to build. As this is not a production array/build, Thin provisioning will save you some disk space.

Beyond those two things, you can typically get an install to work fine with just about anything. My home box is running an unlocked Phenom X3->x4 on a 760/780 chipset (can't remember which). I have an Intel NIC and a Realtek NIC (unused, but the MIB is loaded). And I have a RAID 5 of spinning 250GB disks, and 2 x 2.0TB and 1 x 3.0TB for the MythTV DVR. The ESXi load is solid, the build is stable. No complaints. For your testing you don't really need to invest much of anything other than in disk and maybe some more memory. As long as your platform suits yours needs, use it. If you decide to try to get more out of it, then invest if you aren't able to get enough out of your current platform.

 

[MustISO]

My lab setup is actually a gaming PC that was re-purposed for running ESXi 5.5/6.0. It's running an ASUS Z68 motherboard, Intel i7 2600K and 16GB of RAM. VMs run on SSDs. ESXi is run from USB. I added an Intel dual Gb nic I got from ebay for $15.

For my needs it works great. RAM is cheap now so at some point I may just go with 32GB of ram but I can easily run an AD server or two and a few other servers for whatever it is I'm testing.

 

[nk215]

for a $1000 price point, I would recommend a used Tyan 1366 board with a dual X5570 CPUs on ebay as a combo for usually $100. For another $120-$150, you'll have 72Gig of ram DDR3 registered ECC ram. A good size SSD for VM data storage, a SATA (raid) controller, a few HDD for raw/direct access etc they add up quick.

I don't know the specific on what you want to learn but many serious home labs consist of 2 or more servers for fail-over, vMotion, migrate VM kind of stuffs. So don't put all your $$ into the first server, reserved at least $300 or so for the second server.

In most VM lab environment, you'll be limited by memory. I can tell you that 32Gig is a little on the low side.

Make sure the hardwares you buy support vt-x and vt-d. I strongly recommend server type motherboard. Supermicron, Tyan are good choices.

 

[mvbighead]

for a $1000 price point, I would recommend a used Tyan 1366 board with a dual X5570 CPUs on ebay as a combo for usually $100. For another $120-$150, you'll have 72Gig of ram DDR3 registered ECC ram. A good size SSD for VM data storage, a SATA (raid) controller, a few HDD for raw/direct access etc they add up quick.

I don't know the specific on what you want to learn but many serious home labs consist of 2 or more servers for fail-over, vMotion, migrate VM kind of stuffs. So don't put all your $$ into the first server, reserved at least $300 or so for the second server.

In most VM lab environment, you'll be limited by memory. I can tell you that 32Gig is a little on the low side.

Make sure the hardwares you buy support vt-x and vt-d. I strongly recommend server type motherboard. Supermicron, Tyan are good choices.

A home lab for testing is a FAR cry from needing to support 20 VMs with 4GB of RAM each. Generally speaking, to get the gist and still run a modest home setup, 16-32 GB is adequate. If you give 4GB to each VM, and leave some unused, you could run 7 decently sized VMs. You generally don't HAVE to give each VM that amount unless the workload dictates that they need it. But for a home platform???

And dual CPUs/etc/etc/etc. Yeesh. What kind of workloads are you people doing in a home lab?

For me, my host runs a file server and a DVR backend (MythTV). I tinker with some monitoring software and other things as well, and for the most part, 12GB of RAM gets me by. I don't know what some of you are doing that requires 8 cores and 80+ GB of RAM. You've moved well past testing and into some sort of production workload.

Lastly, if I were to really go the production route, I would simply hop on Ebay and find a used rackmount with a preloaded config that matches my needs:

http://www.ebay.com/itm/Dell-PowerE...620457?hash=item27f84e86e9:g:mPEAAOSwDFNWFDwP

http://www.ebay.com/itm/DELL-POWERE...411090?hash=item2801792f92:g:~4wAAOSwl8NVeGrq

Those are pretty much guaranteed to be on the ESXi compatibility list, and if you select something with 3.5 inch drives, you can load up a lot of spinning disk to have a massive array for storage. Or two separate arrays for VM storage and file storage. And, it's all preassembled so there is very little work to do other than load an OS.

 

[Gunbuster]

I'd look at something like a Dell R515 on ebay. Get a base model, throw in some SSD's and add some ram... Done.