LGA 1150 Haswell Motherboard with proper VT-d support

[soulbadguy]

I will give a try on my laptop tonight , you might right i might be confusing vt-x and vt-d

Ok after some test it seems indeed that i was confusing vt-x abd vt-d somehow... Still reading more about vt-d and hypervisors, it seems that xen is solution to my constant dual booting nightmare...

 

[zir_blazer]

Confirmed working on B85

The guy that posted that, is the same guy that posted videos on Youtube 3 years ago advertising that he managed to do VGA passthrough in Xen, and is what got me interesed in IOMMU virtualization.

 

[zir_blazer]

Another interesing update. I sent a mail to Supermicro support a week or so ago, and they finally responded:

Subject: Intel VT-d support on LGA 1150 Motherboards

I have been interesed in purchasing a LGA 1150 Motherboard for a Haswell build, possibily with a Xeon E3-1245 V3. One of the requisites is that it supports VT-d, so I can use the IOMMU feature to do VGA passthrough with the Xen Hypervisor. There are many consumer Desktops Motherboards from other brands that usually does NOT support VT-d due to the lack of a proper ACPI DMAR table in the BIOS, rendering the feature useless even through there seems to be Hardware support. This is the reason why I started to pay attention to Server oriented Motherboards like yours.

I started to look Supermicro X10SAT Motherboard, that claims in the manual VT-d support. However, one of the reasons why I don't find Server Motherboards convincing is that while I believe they would do the job properly, I will miss all the BIOS arsenal of options to run it out-of-spec (Overclocking/underclocking, overvolting/undervolting the Processor, RAM Timmings, etc), that are very rare to see on a Server Motherboard. There is also another Motherboard similar to the X10SAT, the C7Z87-OCE, which seems to have all those options that I want to have, and also claims VT-d support in the manual, so it seems like a better choice.

However, Intel Ark site says that the Z87 Chipset does not support VT-d, while C226 Chipset does:

http://ark.intel.com/compare/75013,75522

Can Supermicro confirm if VT-d works on the C7Z87-OCE and if that Chipset info is wrong on Intel part?


Besides, accoding to photos, the X10SAT and C7Z87-OCE seems to be based on the same PCB, so they should be very similar. Are there any more feature that doesn't prominently appear on the specifications page that I'm missing by going to one choice against the other, like the BIOS options?

Their response:

Since Z87 chipset does not support VT-d, onboard LAN will not support it either because it is connected to PCH PCIe port. One workaround is to use a VT-d enabled PCIe device and plug it into CPU based PCIe-port on board. Along with a VT-d enabled CPU the above workaround should work per Intel. As for X10SAT, although it shares the same PCB as C7Z87-OCE, will not have overclocking options enabled in the BIOS since that was not what it is designed for.

Basically, what I understand now is that you can get VT-d working on any Chipset, just that you will only be able to virtualize the PCIe slots connected to the Processor and not the ones from the Chipset. This is getting interesing.

[FONT=&quot][/FONT][FONT=&quot][/FONT]

 

[Gaming_Kali]

Taking what you've discussed here, it looks like virtualized gaming may be possible:

Intel 4770 (Vt-d support)
ASRock Z87M Pro4 (confirmed VTd support)
AMD 7970HD (tested working with VGA passthrough)

These three will get you VTd support as well as Xen VGA pass-through. Then you get this possible: Video

Just put games into the Windows 7 OS, stay inside your Linux variant for everything else. Of course you'll need to add plenty of memory supported by that motherboard (listed here), a power supply that supports Haswell with at least 650W (newegg search), and enough hard drive space for both OS's and your gaming.

Guessing this rig will run approx $1,400 USD, without OS/HD's/case/keyboard/mouse.

Missing anything?

 

[mlody]

I do not have any way to confirm this by testing; however, My Asus H87M-E has an option in bios to enable VT-d - strangely it is in a different section then VT-x option.

 

[zir_blazer]

*BUMP*

I'm still deciding between purchasing the Supermicro C7Z87-OCE, that Acmemicro got for 251 U$D (I still have to see how the hell to import it, through), or to wait for the X10SAT that still isn't officially released, through I saw some canadian dealer with it a 300 U$D taking back orders. Anyone knows if these can be got cheaper? Both Motherboards seems to be very rare.
Besides, I'm still thinking regarding if purchasing now, or wait until the fixed versions with the C2 Stepping of the Haswell Chipsets appears on the market. I think I'm going to mail again Supermicro asking about that.

 

[zir_blazer]

*BUMP*

I asked a few days ago to Supermicro support about X10SAT, and they replied in under 1 day. Forgot to post it here:

Subject: X10SAT availability and Stepping C2 for Haswell Chipsets
I have three questions:

1) Is there any official launch date for the Motherboard model X10SAT? When availability is expected? I saw it on some eRetailers, but was a back order product.

2) All the Haswell Chipsets at launch were the C1 Stepping, that had a bug related to USB3 after waking from a sleep power state. Intel was going to fix it in the C2 Stepping, which was to be released this month or next one. At what time I could expect that Supermicro Motherboards with the C2 Stepping Chipsets will be available?

3) How it will be possible to identify if an specific Motherboard uses the C1 or C2 Stepping Chipsets? Will they be identificable by a different model/revision? Can I expect that eRetailers correctly advertises if they have C2 Stepping Motherboards so I know if I am purchaing the fixed one?

Their reply:

Hi,

X10SAT is targeted to be launched in mid -August, there is no C1 version, will launch C2 only.

Extremely useful info. I think I will wait for X10SAT, assuming I can do this.
Only thing that bothers me is that support never tries to give you more info that you asked. If I wanted to buy a C7Z87-OCE instead, I would need to ask again about how to recognize a C2 version.

 

[RyC]

That is awesome news. I tried emailing them asking for a release date for the X10SAT and they didn't respond. I just hope the C2 Haswells come out at the same time so I don't have to wait any longer!

 

[RaiderJ]

This thread has been really informative! I've just recently gotten myself up to speed on VT-x and VT-d. Looking to build an ESXi box that support VT-d so I can pass through a SAS HBA for a ZFS setup (which will then be used as the VMware datastore).

Definitely going to keep my eye on that SuperMicro board.

 

[vailr]

Supermicro C7Z87-OCE (Intel Z87) Motherboard Review
http://www.tweaktown.com/reviews/5645/supermicro-c7z87-oce-intel-z87-motherboard-review/index2.html
Didn't see any mention there of C1 vs. C2 chipset version, however.

 

[zir_blazer]

Supermicro C7Z87-OCE (Intel Z87) Motherboard Review
http://www.tweaktown.com/reviews/5645/supermicro-c7z87-oce-intel-z87-motherboard-review/index2.html
Didn't see any mention there of C1 vs. C2 chipset version, however.

So basically:

* It uses the old BIOS instead of a UEFI one, and many of the typical overclocking options are scattered around. Not very user friendly compared to other modern Motherboards.
* Looked like a solid performer at stock settings, except in one test with the SSD
* It looks very plain compared to every other enthusiast-class Motherboard in existence. Actually, I don't see anything bad with this, I like it for that very reason - I'm tired of seeing Motherboards with ridiculous PCB and slot color combination to "stand out of the crown" and other useless features that add in the cost but nothing in functionality.

I think its weakess point could be price. At the 250 U$D price point that I saw it at AcmeMicro, it has quite a bit of competence that will always outshine it, mainly for the third point. I suppose Newegg should bring it cheaper.
But from a functionality standpoint, it looks like a solid, robust Motherboard, as most people that knows Supermicro seems to expect from them.

 

[zir_blazer]

We're now at mid-August. Looks like there will be some delays.

 

[powerhouse64]

I've been reading this thread and the OPs considerations. Here some comments:

1. It's indeed a messy situation, with some vendors adding to the confusion (particularly Asus).
2. Stay clear of Asus - they support MS Windows only in their desktop boards, and they really don't seem to care about Xen/kvm etc. users. Forget about Asus support on virtualization matters.
3. Asrock has been reported to be a good choice - I made the mistake to buy an Asus board.
4. VT-d or IOMMU is a function of the memory controller. Newer Intel CPUs incorporate the memory controller inside the CPU. BUT, the motherboard BIOS needs to support the VT-d functions. Here is where many manufacturers fail.
5. Even the ark.intel.com website sometimes gets things wrong, so I wouldn't take a "Z87 doesn't support VT-d" as the last word. If you find a mistake, write them a mail or comment and they'll check and fix it.

Here some links that might help:
VT-d compatible motherboards
HOW-TO make dual-boot obsolete using XEN VGA passthrough
{Guide} Create a Gaming Virtual Machine
and KVM VGA-Passthrough using the new vfio-vga support in kernel =>3.9 (for KVM).
Most of the above links are Xen specific, but the hardware info should be good for other PCI / VGA passthrough applications as well.

It's good that you take your time to make sure you got the right hardware since the hardware is really the most critical part. Pay also attention to BIOS releases - new BIOS releases may fix a missing/broken VT-d support, or just the opposite, break VT-d support as (yes again) Asus has demonstrated.

Also, make sure the motherboard vendor supports Linux and/or VT-d and get it in writing.

Good luck!

 

[Maximilian]

Wow bit messy this situation...

http://uk.hardware.info/reviews/444...6x-z87-4x-h87-and-3x-b85-gigabyte-ga-z87-d3hp

21st picture shows an option for VT-D on this gigabyte mobo. They have a 4770k plugged into it though which is why its greyed out, strange that it says enabled though.

Theres 33 haswell mobos with similar bios pics, might shed some light on VT-D support options being in the bios.

 

[powerhouse64]

Wow bit messy this situation...

http://uk.hardware.info/reviews/444...6x-z87-4x-h87-and-3x-b85-gigabyte-ga-z87-d3hp

21st picture shows an option for VT-D on this gigabyte mobo. They have a 4770k plugged into it though which is why its greyed out, strange that it says enabled though.

Theres 33 haswell mobos with similar bios pics, might shed some light on VT-D support options being in the bios.

A VT-d option in the BIOS is good to see but doesn't proof it actually works and/or is supported by the vendor.

For example, a vendor may offer a VT-d option in the BIOS, but a later BIOS update may break support, or the support may be partial or faulty from start. When asking the vendor for a remedy, the vendor can then point to the list of supported OS and if this list includes only Microsoft Windows then you are out of luck! Microsoft doesn't use VT-d, as far as I know.

This is why it is best to contact the vendor and get a written confirmation that VT-d is supported. In addition it would be nice to see vendors committing themselves to support Linux.

Make sure that the vendor's support staff is familiar with virtualization, Linux, etc. Some vendors' support staff has obviously no clew about anything outside the Microsoft realm - stay away from them.

On the positive side is that virtualization and PCI passthrough is starting to catch, as can be seen by the popularity of the forum threads I posted above. There are already many people who made Xen or KVM virtualization with VGA passthrough work, and ditch dual boot setups.

 

[paul878]

This is very interesting, so does anyone have a TESTED combination of hardware that actually works?

 

[powerhouse64]

This is very interesting, so does anyone have a TESTED combination of hardware that actually works?

The links I posted above should give plenty of examples of working hardware. My hardware is Asus Sabertooth X79, i7 3930K (C2 stepping), AMD 6450 graphics card for dom0 (Linux) and Nvidia Quadro 2000 for Windows 7 domU (VM). I'm the author of the Linux Mint how-to, and a number of people have posted successful configurations in the thread. The other threads also contain various configurations that work. The Xen wiki is another source.

 

[rekd0514]

what do you think of this board?

ASRock E3C224-4L
http://www.asrock.com/server/overview.asp?Model=E3C224-4L

 

[powerhouse64]

what do you think of this board?

ASRock E3C224-4L
http://www.asrock.com/server/overview.asp?Model=E3C224-4L

This looks very much like a board for a small, headless server (headless = no keyboard, screen, etc.). So it depends on what you like to use it for. For a Linux / Windows rig such as a desktop computer with Windows gaming VM, you could probably get more bang for the buck with a desktop board and a i7 4770 processor.

I'm unfamiliar with the Intel Xeon series, but http://www.cpubenchmark.net/high_end_cpus.html shows some good results for the more expensive CPUs in that series. The i3 is definitely pretty entry level.

Having said that, this board looks like a good choice for an always-on server (e.g. media and/or file server) that's controlled remotely. So if you plan to build such a server, this might be the perfect choice. Depending on your needs, you could run multiple VMs with different OSes and/or applications, for example a XBMC media center in one VM and a NAS (such as Nas4free http://www.nas4free.org/) in another VM. You could check with ASRock if you can pass through the individual LAN ports to different VMs (PCI passthrough), this way you could make good use of the 4 LAN ports.

Bottom line: This board looks like a safe choice for a Linux / virtualization box. It will cost you more than an ordinary desktop board, but it may be worth the price for your peace of mind.
ASRock seems to have some competent tech support (at least I've read several user reports to that end) so I would send them a mail and ask them for their recommendation. Of course you need to say what you want to do with the PC.

Sorry my answer fell out so long, not just a yes or no.

In any case, before you purchase a board, contact the manufacturer and have him confirm that the board complies with your requirements.

P.S.: Do you want to share your intended configuration and use for the PC?

EDIT: I just saw your previous post here and your description. I'm not familiar with ESXi but the vendor lists it. From what you plan to do it looks like a good choice, but check with ASRock to avoid surprises. I personally prefer open source software / solutions to be on the save side.

 

[rekd0514]

I plan on using it for Esxi like you saw. I just want a board with vt-d for pass through of NICs and Raid cards. This board has IPMI as well.

I figured I would ask in here since this might be another valid choice for those of us that were waiting on the X10SAT if it has comparable features for a lower price. It looks like the board below it is the same with 2 less NICs as an option too.

I was also curious if the included graphics chips on these server boards were good at all or would getting the Xeon with the iGPU included be better. I need decent graphics capability because I plan on recording TV in WMC on a Win7 VM and some possible transcoding, etc..

 

[zir_blazer]

ASRock E3C224-4L uses the C224 Chipset that does NOT allow you to use the Haswell IGP, that's a big difference in the feature list considering how much Haswell die size is dedicated to the GPU. You need a C226 for that. AsRock was going to release a mITX Motherboard with it, the E3C226D2I, but supposedly it was going to be also a C2 Chipset only, reason why it was delayed. Otherwise you have the C226 WS. Most Server Motherboards does not use the C226, it seems almost exclusively reserved for the Workstation ones, limiting choices a lot.

 

[rekd0514]

ASRock E3C224-4L uses the C224 Chipset that does NOT allow you to use the Haswell IGP, that's a big difference in the feature list considering how much Haswell die size is dedicated to the GPU. You need a C226 for that. AsRock was going to release a mITX Motherboard with it, the E3C226D2I, but supposedly it was going to be also a C2 Chipset only, reason why it was delayed. Otherwise you have the C226 WS. Most Server Motherboards does not use the C226, it seems almost exclusively reserved for the Workstation ones, limiting choices a lot.

I'm guessing the built in graphics on most server boards isn't too great since it usually isn't needed.

It looks like I will wait for the X10SAT and get the 1225v3 or 1245v3 to go with it. I already bought 16GB (8GBx2) Samsung M393B1K70DH0-CK0 for the memory.

 

[rekd0514]

http://hardforum.com/showpost.php?p=1040151308&postcount=33

X10SAT is now available, I'm going to wait for a newegg combo with a xeon

 

[zir_blazer]

X10SAT web page was updated. Instead of "Coming Soon", it says "Workstation Motherboard", but the Manual still isn't there. Newegg also has the C7Z87-OCE for 245 U$D, I don't recall they had it some weeks ago. Too bad no one knows if its C1 or C2.

Come on Supermicro. I want to upgrade ASAP.

 

[ykapadia]

This looks very much like a board for a small, headless server (headless = no keyboard, screen, etc.). So it depends on what you like to use it for. For a Linux / Windows rig such as a desktop computer with Windows gaming VM, you could probably get more bang for the buck with a desktop board and a i7 4770 processor.

I'm unfamiliar with the Intel Xeon series, but http://www.cpubenchmark.net/high_end_cpus.html shows some good results for the more expensive CPUs in that series. The i3 is definitely pretty entry level.

Having said that, this board looks like a good choice for an always-on server (e.g. media and/or file server) that's controlled remotely. So if you plan to build such a server, this might be the perfect choice. Depending on your needs, you could run multiple VMs with different OSes and/or applications, for example a XBMC media center in one VM and a NAS (such as Nas4free http://www.nas4free.org/) in another VM. You could check with ASRock if you can pass through the individual LAN ports to different VMs (PCI passthrough), this way you could make good use of the 4 LAN ports.

Bottom line: This board looks like a safe choice for a Linux / virtualization box. It will cost you more than an ordinary desktop board, but it may be worth the price for your peace of mind.
ASRock seems to have some competent tech support (at least I've read several user reports to that end) so I would send them a mail and ask them for their recommendation. Of course you need to say what you want to do with the PC.

Sorry my answer fell out so long, not just a yes or no.

In any case, before you purchase a board, contact the manufacturer and have him confirm that the board complies with your requirements.

P.S.: Do you want to share your intended configuration and use for the PC?

EDIT: I just saw your previous post here and your description. I'm not familiar with ESXi but the vendor lists it. From what you plan to do it looks like a good choice, but check with ASRock to avoid surprises. I personally prefer open source software / solutions to be on the save side.

Yes, each network port of E3C224-4L(Intel i210) can support pass through individual VM.