I have recently built a replacement home server that has superseded my old trusty HP N54L. I would like to say thanks to all who patiently gave input, suggestions and help etc in this thread: Mobo and other HW suggestions wanted for new OMV box with ECC as the finished article would not have been possible without them.
Project Goals:
1. A media server that will easily stream 4K to several devices in the house at once
2. A place for backups from our laptops that will protect the integrity of family photos/videos
3. Wife Seal of Approval
4. Other things in the future.
Wife Seal of Approval?
We all know that when we want to buy a new toy that we have to think about the Wife Approval Factor. Geek girls - you don't have this problem, because when you say to your husband/partner that you want to build a l337 media server, he's probably drooling already and telling his mates how his Mrs is awesome. However, any guy who has set out on a project in the name of making life easier through IT that will benefit the house will have been stung at some point. My wife probably gets fed up of me talking about my latest project ideas, but she's incredibly tolerant and occasionally gives a lot of helpful input and helps with varying processes (although I don't think she'll ever forgive me for the Xbox360 cooling project that was a lot more hassle than it needed to be). She doesn't usually mind when I want to invest in a project but:
1. It must 'just work' - when the wife wants her music or movies or whatever on her device of choice, it must work. No one likes investing in something that doesn't do what it says.
2. No upgrading and piddling around with hardware every 6 months - RAM upgrades and scheduled software maintenance is allowed, usually when she is out with the girls
In fact, my wife has been an active part in this project and has helped with the choosing of certain components and helped with assembling the build as well. For more information on Wife Approval Factor, please see this wiki: https://en.wikipedia.org/wiki/Wife_acceptance_factor
Without further delay, here is the finished article:
Please note that the WAF Wiki article references the use of 'appealing colors' to gain WAF - My wife chose this chassis.
The build:
CPU: Intel Xeon E5-2695v3 [preowned]
CPU Cooler: Corsair H105
Thermal Compound: Arctic MX4
Mobo: ASRock X99 WS
GPU: Nvidia GeForce 9500GT
RAM: 32GB DDR4 ECC (2 x Crucial 16GB DDR4-2133 RDIMM (CT16G4RFD4213))
PSU: Seasonic Platinum SS-400FL2 400w Modular Fanless
Chassis: Thermaltake Core V71
Drives (RAIDZ2): 8 x Seagate ST4000DM000
OS drive: Kingston SSDnow V300 120GB
SATA cables: Blue flat for OS SSD, 8 x Black rounded for array drives (generic, no links)
CPU - Xeon E5-2695v3:
The main decision to use a Xeon CPU was ECC support coupled with the power to last. The use of the ZFS file system pretty much requires ECC, so my choices were: Avoton, Some of the i3's and Xeons (or AMD but I don't like them despite the N54L having an AMD). Being that I had in mind that I wanted this system to last a reasonable amount of time, the Xeons ended up being the solid choice. This then gave me the problem of budget. Initially I was considering an E3 series Xeon or a lower end E5, but a friend of mine who sells refurbished components sold me an E5-2695v3 for £350 and a crate of beer. With 14 cores at a base frequency of 2.3GHz up to 3.3Ghz it's one of the most powerful CPUs this board can take and is unlikely to need changing for the life of the build.
CPU Cooler: Corsair H105
When we decided on our motherboard choice (see below) we noticed that, unlike a lot of server boards, it has a Square ILM (pitch). This opened up the use of pre-built CPU water coolers and after looking through a lot of review sites the wife and I decided on the Corsair H105 due to it's extreme cooling performance and reasonable price. We were initially concerned about noise, due to one YouTube video showing it to be a little loud, but we don't regret the choice - it's a fantastic unit and very quiet even under load. My only complaint of Corsair was that until recently, they had no UK RMA address in the event things go wrong. This has now changed.
Thermal Compound: Arctic MX4
We opted to remove the stock Corsair paste in favour of MX4 which I totally love. We're huge gamers and our Xbox's and PlayStation's have all been re-pasted with MX4 and we've never had any problems with them. The use of MX4 was certainly a good choice when thinking about the Xeon CPU, the highest I've seen it get to is 35C on a hot day. Great paste, great price, great performance - what more could you want?
Motherboard: ASRock X99 WS
This was the hardest part of the system to settle on and, as those who were involved in the honing process will know, ended up being a small project in itself. Again due to ZFS, we needed to find a board that supported ECC. ECC only works if the motherboard, CPU and RAM all support it. Originally, as mentioned, I was looking at an Avoton all in one board because oddly enough those little Atoms support ECC. We decided against this as we felt that the CPU would probably need upgrading before the build was retired/reassigned. Being that they are integrated BGA units, the board would need to be replaced as well. We initially chose to go with a Supermicro X10SRA-F which was a reasonable £282. Initially things appeared to be good, but the board suffered from some IPMI bugs which Supermicro were unable to fix. When we sent the unit back for RMA we hit another problem. We had already committed to the Corsair H105 and didn't want to change it. This completely limits us to boards with a square pitch. The only other server boards with a square pitch that ticked all the other boxes were the ASUS Z10PA-U8 and Z10PA-U8/10G-2S. I've had a handful of negative experiences with Asus, so these were not an option. At this point I was concerned because I wasn't sure if it was actually going to be possible to keep the Corsair. However, checking through some of ASRock's highend workstation boards I noticed that the X99s on their site list support for ECC. From talking with others it's been noticed that sometimes a manufacturer will say that the board supports ECC memory because it will run with it, yet the ECC feature is never enabled. Being that ECC is notoriously difficult to prove if it's actually enabled, I wanted to make sure so I dropped ASRock an email. They confirmed that their X99 boards that list ECC do work with, and enable ECC providing a Xeon CPU and ECC memory is used. Pleased with this response, we went for the ASRock X99 WS from Amazon for £254.99 which looks stunning covered in blue heat sinks and has an absolutely amazing UEFI. Unfortunately the board doesn't have IPMI and integrated graphics, but the many benefits of this board and the excellent build quality have won through. In terms of Debian support, everything was detected and no restricted firmware was needed.
Update: 13/7/2015: There is now a way to verify if ECC is working but it uses Windows. See below.