New OpenMediaVault user (long time Debian user) here. I chose OpenMediaVault due to that familiarity, not to mention that my day job involves various flavors of Linux so dropping to the CLI and doing things there is not an issue for me.
I've had other NAS's in the past ranging from a HPN40L to a Synology one (until it recently died). Now fed-up of propriety hardware I decided to build my own (the power supply is still en-route).
Hardware:
- Case I've gone for a 8 bay chassis from IPCRYM off Aliexpress
- Replacing the 60mm fan : Noctua NF-A6x25
- Mobo / CPU / Ram : All from Ebay
- The motherboard is a Supermicro X10SL7-F
- Replacing the Heatsink : Noctua NH-L9i Intel-only Low Profile Quiet CPU Cooler
- The CPU is a low power Xeon E3-1230 (E3-1230L v3)
- 32GB DDR3 1600MT
- The motherboard is a Supermicro X10SL7-F
- PSU: Silverstone Flex FX350
- OS Disk : Corsair Force Series™ LS 60GB SATA 3 6Gb/s SSD
- Extras:
- SSD Cache : Ortial OM-350 128GB SATA III M.2 SSD
- Storage : 8x WD Red Pro
I purchased the USB-3 card for future expansion, when I was looking for a reasonable one I found one that had a SSD slot (B-Key) so thought it useful for later. The OS SSD I had in my drawer and it hadn't been used so I figured it should should be a good / reliable enough drive for the OS. The Cache drive was on a "today only" deal from a UK website and I was able to get it for less than £15.
I had initially ordered a both the case and PSU from Aliexpress on the 28th April. The case arrived yesterday 20th June. I was aware the case might take some time, however the original PSU ordered has not arrived and the tracking is non-existant on that order even though it had a 15 day guaranteed delivery, so that one is in dispute. I've used AliExpress a bunch before and it's generally been quite good. I've ordered the Silverstone Flex off Ebay, a seller there had what I considered to be a good deal, it should be arriving in the next few days. Right now the whole thing is running off a desktop PSU so looks a little messy.
The motherboard looks to have already been flashed to support "IT" mode on the 2308 SAS controller so that's one less task for me 
. "IT" mode allows SATA pass through, aka Linux can see the individual drives instead of the RAID controller getting in the way. While I'm a fan of high end raid controllers, I find the low end raid controllers to be rather limiting and prefer software raid in their place. The motherboard / CPU / RAM came with a stock lump of copper cooler, which is good if the case has ducting for the airflow to pass over the CPU cooler. Even though the CPU has a 25W TDP, my case does not have ducting, I don't have a 3D printer, the will to cut / fold / glue thinner pieces of plastic, not to mention all the SATA data cables are right behind the fan, so I'm swapping the cooler for a different one. As I was buying a new CPU cooler from QuietPC I added the 60mm case fan, the stock case fan has 2 wires going to a molex (old HDD power) connector and I thought I'd upgrade to one that would be quieter and have PWM so its speed can be controlled.
With my previous NAS being a Synology DS1815 and it having an Atom CPU I knew the mobo and cpu did not have to be a powerhouse. I chose this specific motherboard after looking at many more current ones, but as soon as you want multiple NIC's the selection starts going down, more than a couple of SATA ports further narrows the selection and for each the price goes up. I sort of stumbled across this mobo, and when you factor in the motherboard has full lights out management with embedded KVM and it came with a CPU and RAM, it kinda chose itself. I'm not planning on doing alot that isn't directly storage related so I don't need a faster CPU. I have another computer I use for VM's as well as a few RaspberryPi's so they will be doing tasks that others might put onto their NAS. Per above I'm still building it out, but looking at the current power it's less than 30watts (including the spinning drives) so I'm happy in that regard.
The Data HDD's are moving over from my previous NAS, however the data is temporarily going elsewhere while I reconfigure the storage.
Configuration:
The main purpose for the NAS is media storage, mostly video taken during scuba diving, however there is also the usual mix of personal files sync'd from my laptop etc. There's also a few other hosts on my network which access the storage via NFS. A local git server is also useful as well.
- Network : The two NIC's on the mobo have are configured in an 802.3ad bond.
- The bond is a trunk with a 9000 MTU and multiple 802.1q tagged vlans.
- It would be nice if we could rename the 802.1q interfaces through the webUI but I guess that's something I'll need to look at in the netplan config (I'm used to using /etc/network/interfaces and it's .d sub-directory).
- The bond is a trunk with a 9000 MTU and multiple 802.1q tagged vlans.
- Storage : I'm sure this will be the most contentious point for many
- Over the years the biggest lesson (Golden Rule) I've learnt from RAID is NEVER put the whole (raw) disk into an array. Always partition the disk and shave a few hundred MB (or even a few GB for larger disks) off the total size. The exception to that rule is if you have a commercial appliance such as a Netapp and you have a support agreement which includes the disks. The reason for this is consumer grade disks (ones that you would go and buy through normal channels) vary in their exact size. When a drive fails (which it will do eventually) it will obviously need to be replaced. I've had atleast one instance with the same type of drive from the same manufacturer eg 4T from Seagate, has been smaller than the drive it was to replace. I'm sure many on this forum will know you should only replace drives with the same part number. Eg A st4000ne001-2ma101 may not be able to replace a st4000ne001-3cr101 (putting a side any CMR/SMR issues)
- An advantage of using partitions is that you can grow your array over time by adding a new larger disk you can create a partition that is the "current" size, and the new disk can have a second partition which is then joined into a new array. The two arrays are then joined eg by RAID0, LVM or similar.
- I have partitioned my disks. I have added 1 partition from each disk into a RAID 6 array. I'm happy to take the performance hit and I don't want to get a tape drive and have to deal with all of that.
- I'm using bcache with the Ortial 128GB SSD - I'm using writeback (I might change that in the future)
- I've decided on BTRFS for the filesystem.
- I'm after the ability to snapshot.
- I might use it's sub-volumes, but this is undecided.
- I've been using it for years on my laptop and so I'm happy that it's stable enough for my purposes.
- NCQ's are disabled (while the array is building), I might put them back at a reduced level later
- (as root) apt update && apt install -y sysfsutils && for i in $(echo {a..g}); do echo "block/sd${i}/device/queue_depth=1" >> /etc/sysfs.d/disk-ncq.conf;done && systemctl restart sysfsutils
- Over the years the biggest lesson (Golden Rule) I've learnt from RAID is NEVER put the whole (raw) disk into an array. Always partition the disk and shave a few hundred MB (or even a few GB for larger disks) off the total size. The exception to that rule is if you have a commercial appliance such as a Netapp and you have a support agreement which includes the disks. The reason for this is consumer grade disks (ones that you would go and buy through normal channels) vary in their exact size. When a drive fails (which it will do eventually) it will obviously need to be replaced. I've had atleast one instance with the same type of drive from the same manufacturer eg 4T from Seagate, has been smaller than the drive it was to replace. I'm sure many on this forum will know you should only replace drives with the same part number. Eg A st4000ne001-2ma101 may not be able to replace a st4000ne001-3cr101 (putting a side any CMR/SMR issues)
- Other:
- Nginx / SSL : I'm using Lets Encrypt with either certbot or cert-manager both making use of the dns-rfc2136 plugin. Using DNS rather than HTTP to validate the certs means I can have SSL certs on my local network without having to open port 80 to the net. Handy when my ISP insists on "Carrier-grade NAT" due to IPv4 shortage.
- I have enough ram so I've dropped the swappiness to 10. The system will swap to avoid an out of memory (oom-killer) but it will be little more than that.
- (as root) echo "vm.swappiness = 10" >> /etc/sysctl.d/88-swappiness.conf && sysctl --system
- I've added `noatime` to the root volume params. - I'm not going all the way that some have to reduce writes to the OS disk, but I am doing some basics.
- Installed
- lm-sensors (and re-scanned)
- cpufreqd I'd rather have something actively slowing the cpu if the thermals start getting high.
- The Noctua cpu cooler on it's way to me and when I first started the array (and disabled NCQ's) the CPU got hot and triggered some c-states. I had left it to go to the gym and when I got home I found it beeping and my better half
with noise cancelling headphones on and a fleece over her head multiple times to try and attenuate the sound (oops). It didn't take me long to shut down the NAS, find a USB fan for the next day and order the replacement CPU cooler.
- The Noctua cpu cooler on it's way to me and when I first started the array (and disabled NCQ's) the CPU got hot and triggered some c-states. I had left it to go to the gym and when I got home I found it beeping and my better half
I've chosen the RAID setup based on my experiences so if anyone can think of ways to improve it, any constructive feedback is appreciated, however "That's rubbish" or "I'd never do it that way" is likely to fall on deaf ears.
Any questions is more than welcome.
In terms of the pictures:
First 3 from AliExpress :
From the ebay seller (for the mobo, cpu & ram)
From Me, Case :
Current CPU cooling (until the replacement cpu cooler arrives in a day or two)
Edit - Additional pics
