What is OMVs "recommended/native" storage solution? (setting up a brand new storage)

I've always liked (and still do) mdadm raid (depending on your disk availability, but I'm running raid 10 - 4 drives...two would be a simple mirror), with lvm on top (to give create/grow/shrink/snapshot), and ext4 as the fs. Solid, dependable, boring. Well established (and supported by omv) stuff that works.

Zitat von tetricky

I've always liked (and still do) mdadm raid (depending on your disk availability, but I'm running raid 10 - 4 drives...two would be a simple mirror), with lvm on top (to give create/grow/shrink/snapshot), and ext4 as the fs. Solid, dependable, boring. Well established (and supported by omv) stuff that works.

I personally use mdadm raid 5 and a mdadm raid 1, both with an xfs file system. I have no need for create/grow/shrink of volumes, so I don’t use lvm, and of course xfs does not support shrink anyway, but it is a faster filesystem than ext4 for larger files, and it supports parallel I/O so it’s better for multiple users/services accessing the same filesystem.

I agree that simple is usually better, but zfs is a compelling option too since it is a combination filesystem and raid in one solution. I have used it before in test setups of freenas and true nas. Even though others are using it, I wish it had native support in the standard Linux kernel instead of having to use the proxmox kernel. I would probably feel more comfortable with it then.

I agree that xfs is faster for bigger files. My general use case tends to be lots of little files, and I find ext4 a better fit for that. when I last tried zfs the memory overhead was onerous to get the benefits of it, and it felt a bit clunky to use....Although sitting at the bottom of the rack next to me is an old proliant ml50 with a zfs array in it. It's not been turned on for a few years. I'm currently using it as a shelf for the microservers. It's too loud to sit next to.

I don't use a lot of media files...I muck about with various services that require lots of little config and data/db files....and containers...so lvm's ability to create and remove logical volumes and shrink/grow where necessary is useful for me. Also document management and creation...so ext4 beats xfs for me.

Your comments are entirely valid though...and herein lies the problem. There isn't really a one size fits all solution. What I do fits my use case...but other people are not me....

molnart The answer to your question is that it's horses for courses. While making a choice don’t lose sight of the fact that the primary reason for RAID is uptime and it’s no substitute for proper backups. It’s always going to be a balance between capacity, performance and redundancy level.

You know enough about mergerfs + SnapRaid to understand its pros & cons, so that leaves, MD RAID, BRTFS and ZFS.

Plenty of people have stuck with MD RAID for better or worse, but the number of forum posts re: failed/inactive arrays shows people did/do not understand how to deal with failures and other admin tasks.

The inclusion of BRTFS support first in OMV6 and then OMV7 is meant to offer something better. In many ways it does, but there are gaps compared to MD RAID. There is no detailed info available via the WebUI for BTRFS filesystems spanning multiple devices. BTRFS has no in-built monitor daemon to alert the user via notifications of failed devices. OMV only offers details about device stats, there is no additional functionality to monitor the logs in real-time for BTRFS problems. As BTRFS tends to go read-only when problems occur you would get a generic email when it fails to mount because one or more devices are missing on a re-boot. You need to resort to the CLI to deal with “missing” devices, BTRFS balance, etc.

Even if you put the 50% storage efficiency of BTRFS RAID 1/10 to one-side, it’s hard to find a valid argument to use BTRFS RAID for more than 3 or 4 drives on an array that can only sustain one device failure and does not guarantee to mount read/write and so provide the Uptime associated with other RAID implementations that can mount in degraded mode.

Lastly, if you say Uptime isn’t really the primary concern of home users as it is really about combining a number of drives to make a pool and BTRFS makes it really easy to add disks to an existing pool, then even here you can be caught out. For example, add a 8TB to a BTRFS RAID1 with 2 x 2TB drives and you only get to use 4TB of that 8TB drive.

As you are contemplating using ZFS, you must already be aware of its advantages over BTRFS and how it can be used in OMV. It’s less tightly integrated in OMV, automated scheduled scrubs are easily covered and automated snapshot admin can be done via various 3^rd party scripts/progs. Most CLI ZFS commands are straightforward and relatively intuitive, such as replacing a failed drive. Unlike BRTFS, you have a “rollback” command. More importantly a degraded ZFS pool will mount read/write and so maintain uptime. ZFS has a comprehensive monitor system – ZED – so you get prompt notification of problems.

To answer BernH point, it’s not obligatory to use a pve kernel with ZFS. The standard debian kernel works well enough with the zfs-dkms package. Debian kernels do not change frequently, so the DKMS build times is not too much of a problem. You are however stuck with zfs 2.1.11 unless you choose to install the debian backport kernel and associated packages which will currently give you zfs 2.2.3 which is on par with pve kernel >= 6.5

tetricky Mentioned on ZFS "the memory overhead was onerous to get the benefits of it .." Without knowing the details it's hard to say anything precise here about a comparison with ext4. ZFS was not designed with speed as it's top priority and its need for memory is often overstated nor is memory the only factor to determine performance. For example, a workload that's heavy on random IOPS is not a great match for a raidz1/2 pool. A workload that generates a lot of sync writes on a pool made up of HDDs may not perform well without a separate log device even if a mirrored vdevs are used.

molnart You didn’t say how many disks, or what size, you intend to use, but for a strip of raidz1 it must be at least six. If you’re doing mostly sequential read/write I would have thought a single RAIDZ2 would be a safer bet.

I could go on a lot about OMV versus TrueNAS Scale, but suffice to say I’d still pick CORE over SCALE any day if ZFS storage was the main use of the NAS. But for home use OMV wins over SCALE’s closed appliance model, offering a sensible implementation of docker-compose and VM/LXC with overall a much greater flexibilty of use.

Thanks for the clarification on zfs in the standard kernel. I was aware that previous kernels used in the last couple of Debian versions had rudimentary support and required the proxmox kernel for better support but I was not aware it had improved a bit in the standard kernels. I should have been a little clearer in my statement, but regardless, I just learned something new too.

I don't have the answer, but there is a question often overlooked.

What happens if the machine around the drives dies...not the drives themselves?

I know with mdadm (software raid) it's relatively easy to rebuild array on a complete separate machine. I have had to do that....I have also been privy to a situation where a proprietary raid controller card has died and taken all the data with it (settings stored on the card itself, and not recoverable from even an identical card).