Prior to OMV I have never used a dedicated NAS OS before. I've been through several iterations of dedicated servers running arch, debian or ubuntu and have always configured my services manually. I have now seen the light!
Case: Silverstone DS380
This all started when I fell for the Silverstone DS380 case. It's an absolute beaut and I had to have it.
Motherboard: ASRock C2750D4i
Mini-ITX, 12 SATA ports, 8 Core 2.4GHz Intel BayTrail in a 20watt SOC (passively cooled), 1xPCIe 8X slot, 2 gigabit NICs.
The DS380 has been paired up with the ASRock C2750D4I on several tech sites, but when I first started I thought it was crazy to spend $400 on a motherboard - especially given the mixed reviews and the fact that I am not a big fan of ASRock (bad previous experiences). I tried to find better combinations of pci-e sata cards and mini-itx boards and no matter which way I turned I kept coming back to the ASRock board. IPMI was the deciding factor.
Memory: Crucial 16GB Kit (8GBx2) DDR3L 1600MT/s (PC3-12800) DR x8 ECC UDIMM 240-Pin Server Memory CT2KIT102472BD160B
I didn't really need ECC memory, but the fact that I could use it and that I use snapshot raid sold me.
OS: OCZ Vertex Plus 120GB - One of the most condemned SSDs. I've abused it for 3 years and it still performs decently.
4*Seagate Barracuda ST3000DM001 - Terrible reviews but I have been using them for two years with no problems. Update: 3 out of 4 died after 2 years 4 months.
2*Toshiba 3TB HDKPC08 - Brand new, slower than the Barracudas.
Expansion Card: PCI-E: Generic Low Profile USB3.0 and eSATA card. (Amazon B00F920IYW).
The DS380 front panel requires a USB3.0 header which the C2750 lacks (I thought they were made for each other!). This card provides the 20 pin header and extra USB ports on the back (the C2750 only has 2). eSATA is provided via passthrough from one of the internal SATA ports.
Virtualization: KVM via Ubuntu Server 14.04
I wanted to keep NAS functions separate from other server duties so I needed to maintain separate VMs. I am new to type 1 (hypervisor) virtualization and spent some time researching ESXI, XEN and KVM. In the end I settled on KVM and have managed to figure everything out so far. I found that KVM is often treated/confused as type 2 virtualization due to the install/management mechanism. As I understand it the initial install OS (in my case Ubuntu Server) is not the host, but rather a client on the hypervisor with management hooks installed.
NAS OS: OpenMediaVault 0.5.48
I installed OMV on an 8GB volume on the SSD using virtio. I am concerned this volume may be too small. I really have been blown away with the ease with which I can configure services, for example CIFS and AFP compared to my old process of manually editing the config files. I was up and running in hours rather than days. Performance has been great.
Storage: Direct Access
I have passed the drives directly through to OMV which KVM advices against. I wanted to be able to pull the drives and view the partitions on remote machines without having to worry about volumes inside KVM images. Maybe LVM makes this irrelevant? Initially I passed through the raw device mapping (e.g. /dev/sdb) but a small disaster after I pulled a drive made me change to use the /dev/disk/by-id mapping.
Comments and Questions
So far the system has been rock solid. Due to some reports of the Marvel controllers dropping out under load I dd'd /dev/zero to all the drives at the same time for a while and didn't see any issues. Aside from that I have made 100+ gigabyte transfers across the network onto the NAS without any failures.
I had issues with the RAM slots. Perhaps one channel (blue?) has to be populated before the other will work, but it took a little swapping around before the system successfully booted (it's been solid through multiple power cycles since).
IPMI is a fantastic feature and I can't believe I never knew it existed. My server is hidden away far from any display, so the ability to remote control was particularly useful when I managed to screw up my "/etc/network/interfaces" and the server had no network access.
I don't get the same performance writing to disk on OMV on KVM as I do in the Ubuntu OS from which I installed KVM. We are looking at 180MB/s versus 140-160MB/s. This isn't a huge deal but I was under the impression the performance would be closer to native using virtio. I have Cache and IO mode set to default.
I haven't looked into enabling drive spin-down, they don't spin down automatically.
I was really surprised at how quiet this thing is. I think you could be comfortable using this as an HTPC in your living room.
Is an 8GB OMV partition enough?
Am I correct in thinking the SnapRAID plugin requires manual Syncs?
Despite the fact there isn't much airflow and there are 6 hot drives spinning away I haven't seen the CPU temperature go above 44°C (under significant load). With 2 VMs running it idles at 36°C. I am very impressed with avoton/baytrail.