(This is a really long one...thanks Adderall! edit: apparently my post is too long and is over 10k character )
I've been a datahoarder for about 15+ years now. I've pretty much ran it all over the years: I started with Linux software RAID and used that on Arch and various other distros; then I discovered FreeNAS 9.3 and ZFS....it consumed me....I spent hours and days reading whitepapers and posts about how ZFS works. I built a small NAS (I had been using my desktop as my server up until this point) which held 6 HDDs and installed FreeNAS on it and created my first pool. I quickly filled that up (like within a few weeks haha) and built a larger one also using FreeNAS. My current server has a Threadripper 2970WX (24 cores @ 3 GHz), 128 GB DDR4 ECC, 1.6 KW PSU, 2x 10G NICs, 8x 8 TB drives, 8x 18 TB drives, and 12 M.2 NVME drives, totaling over 200 TB. The problem is I'm sucker for a nice UI and the UI for FreeNAS sucked. It was also based on FreeBSD 9.3, and I was used to a bleeding edge Linux distro (Arch), so the lack of apps packaged by default, and the PITA of using the Ports collection (being forced to compile everything under the sun), I ran screaming back to Arch with what was ZFSonLinux at that point.
I tried other storage focused-OSes like unRAID (nice GUI and was the only one at the time that allowed me to pass through an Nvidia GPU without getting a code 43) but was put off by the limit of paying for more than 8 drives, by this point I had at least 8 drives, probably 10, so I quickly kicked unRAID to the curb after the trial period was over. I went back to Arch and just used the CLI to manage ZFS, which was getting quite cumbersome when you have 10 or more disks, everything else was running fine though. I had heard about OMV and how it supported ZFS and gave it a shot, but the GUI was barebones and kinda ugly IMO (this was around 2015), and I think ZFS support was pretty much "install our plugin and manage it yourself via the CLI". I was also kinda put off by how it was essentially just a webserver that was running on top of Debian (noob me thought this was a bad thing haha). I had given Proxmox a try as well, but that was even worse since it's main function is as a hypervisor. ZFS had to be managed purely from the CLI.
I had a huge paragraph here about how I've tried every version of FreeNAS/TrueNAS since 9.3 over the years and I've had issues with every one of them. Either simply because it was BSD and I was used to using Linux; internal issues amongst the development team and the community; or simply because the UI was way too cumbersome and they prevented you from doing anything via the CLI, unless it was via their clunky shell. In the end I jumped ship back to Arch every time, even after trying "server-focused" distros like Debian and CentOS.
About 2 weeks ago I got 4 new 18 TB drives and slapped them in my system...and then everything went to hell. I added them in a RAIDZ1 vdev to my pool that already had 4x 18 TB in RAIDZ1 and hadn't been giving me issues for weeks (I bought a no-name 4U server case with a backplane off of ebay in October and I think the backplane is garbage, I've had IO errors randomly since I got it). Within a few hours I'm getting thousands of IO errors on the new drives, the older drives are fine. My other pool of 8x 8 TBs was error free as well. Can't find the source of the issue (still trying to narrow it down to bad cables, bad drives, bad HBA, etc... The pool with the 18 TBs is seemingly fine outside of the case, directly connected to the HBA, so it seems to point to the backplane being the issue). I clear the errors, it attempts to resilver the drives, thousands of errors pop up again and it freezes the pool due to tons of IO errors. After some more troubleshooting to no avail I shut it down, disconnect the drives and reconnect them, and boot it back up. This time it's missing the pool with all my 8 TB drives in it (which had been fine for a while), I attempt to import it and get a lovely "Can't import pool tank. Restore the pool from a backup or destroy the pool" WTF?!?!? It was perfectly fine this whole time, and the pool with literally thousands of errors on multiple drives imports without issues...but the pool without a single error is suddenly FUBAR?! I attempted a few things, but then kissed the data goodbye (it's pretty easy to get back).
Frustrated by the constant issues with ZFS over the past decade I was torn on what to do. I clearly love Arch and built my server around ZFS, and in about the past two years a project called Poolsman has been created which is a closed-source plugin for Cockpit which allows you to manage ZFS from a web UI. It's $80/year for a license (I'll happily pay for a license to help me manage my ZFS pools but won't pay for unRAID...go figure haha) The only problem I have with it is the development is slower than molasses in Alaska during the winter. They only added the ability to create a pool via the UI about 3-4 months ago, you can't do any dataset management. 45 Drives Houston also exists, but I haven't had much luck with it.
Not wanting to pay for unRAID and knowing that Proxmox isn't an option, I decided to give OMV another try.....Wow, what a nice OS it has turned into! A beautiful webUI and tons of functionality out of the box, and if that's not enough just install a plugin. If those aren't enough, then install the OMV-extras repo which has a bunch more! Also I love how there are a bunch of filesystems that are supported and you're not locked into on particular technology like you are with unRAID and TrueNAS. I've been wanting to give BcacheFS a try, I could also just go the easy route of MergerFS and SnapRAID. The only problem I have is apparently the kernel is too old to support the newest ZFS feature flags. Of course I upgraded my feature flags like two weeks ago on TrueNAS because it told me new ones were available. So, if I wanna use OMV I'm gonna have to boot into Arch and copy over the data to another mass storage medium and then recreate it either with ZFS or another technology. I also love how you can simply install the Proxmox kernel (since OMV is also Debian) and essentially have OMV and Proxmox in one OS! It's great that standard KVM is available as well!
I'm sold!