I received a question about the motherboards of my two dual-socket boxes, and I thought I put the answer into this thread:
These were my very first and so far only 2P builds, meaning that I have no experience beyond these. (I had occasional limited access to 2P systems before, but never had to deal with their hardware or system-level software aspects.)
I chose Supermicro because I already had good experience with an X10SAE for my Haswell Xeon E3 as my primary home desktop. Second, I chose X10DAX because I needed good CPU clocks from the Xeon E5s. X10DAX is among the "
Hyper-Speed" line of Supermicro boards. This line features (1.) some sort of turbo enhancement, (2.) BCLK overclocking, VRMs which are capable to support the power draw of 1. and 2., and (3.) latency-optimized firmware.
Of these features, I was keen on (1.) and on the VRMs to go with it. I have not found any proper documentation about what this "Hyper-Turbo" feature is really doing, but I hoped that it would drive the Xeons at all-core turbo clock all the time when under full load. This turned out to be true. I have never seen it go below all-core turbo, except on idle cores. (Fewer-core turbo clocks are occasionally applied provided there is only a partial load, but this is not important to me on these boxes.)
However, it could be that other, cheaper boards can permanently hold all-core turbo too. But I have no idea how widespread this feature is. Maybe it is even standard? I am not just a 2P virgin, but a Xeon E5 virgin as well.
As for features (2.) and (3.): I have not experimented with BCLK overclocking yet, and don't really plan to. Low-latency firmware is good to have, but I don't use these systems as a desktop or for high-frequency stock trading or something like that.
There is no BMC/VGA on the board, i.e. it needs a graphics card. But any DCer worth its salt would put some of those PCIe slots to good use for GPU crunching goodness anyway. (I only have a low-wattage, passively cooled, hand-me-down GPU in there which is only supposed to display the BIOS, and rarely a desktop.) Also, there is no storage controller besides the one in the PCH, and just run-of-the-mill dual 1G networking. The upside of the lack of extra controllers onboard is that POST times are fairly low, comparable with the lowest POST times among X99 boards.
Thermal sensors etc. are rather sparse, at least with Linux. I get to see only one temperature probe besides CPU temperatures, and I have no idea where this probe is located.
The board has several PWM-enabled fan headers, which I like. Fan control options are mostly absent; actually, I don't remember whether there are any options at all, and can't check right now. As far as I can tell, all PWM headers are driven at the same level, dependent on the hotter of the two CPUs. But I am very satisfied with the BIOS's fan curve, in combination with the heat sinks, CPU fans, and case fans that I chose.
Ah, I forgot: Another major factor why I chose X10DAX was because two Noctua NH-D15S fit on it, blowing upwards, straight out of the fully-meshed top side of the computer case. This avoids exhaust from one cooler getting into the intake of the other. In combination with the plentiful radiator surface of NH-D15s, and passive GPU, it makes for a super-quiet system. These two boxes could IMO be used even in a bedroom, both running at full CPU load all day and night. (I have no place to put loud hardware.) The case is Corsair Carbide 400Q; tray modded for SSI EEB layout; front fascia permanently removed for easier cleaning of the front fan filter. (Function over form; case chosen for its compact size. Making it SSI EEB compatible can be a PITA depending on one's craftsmanship, which I for one mostly lack.)
RAM support:
I went for registered ECC RAM. These boxes are supposed to perform week-long computations if needed, and I have little use for bit flip errors during such a computation... The X10DAX manual and QVL list
only mention support for RDIMM (registered), and LRDIMM (load reduced) modules. Supported RAM speeds depend on the CPU generation (Haswell E5-26xx v3, versus Broadwell E5-26xx v4), and in case of RDIMM on the number of populated slots per channel.
I do not know whether unregistered DIMMs could work, have not tried, and currently do not have spare modules to try.
Final note: When I built these two 2P boxes, I had no idea that I would get into distributed computing one day. As they were built for multi-core CPU computing, they happen to work quite well for DC applications too, I believe with good performance per Watt, but obviously not on a performance per dollar level anywhere near a more typical DC enthusiast system.