How best to use BTRFS for both OS and Data in OMV6?

chente · Mar 16th 2023

Quote from Krisbee

https://wiki.tnonline.net/w/Btrfs/Profiles - DUP mode protects against data or metadata corruption, but not disk failures.

Alright. Looking at that link carefully I understand. My brain was not fooling me. To have protection against data corruption it is necessary to lose half of the storage space. So that statement is half true. With that system you lose the same capacity as with a Raid1. To do that it is preferable to create a Raid1 and for the same money you also protect yourself from a disk failure. Also don't stress the disk so much.

So for now I'll stay as I am, if I'm forced to format I'll probably do it in btrfs just for better compatibility with OMV, but nothing else.

Thanks for the info anyway.

Krisbee · Mar 16th 2023

Quote from chente

From that approach this conversation no longer makes sense, because it would not make sense to use raidz1.

If I lose the second drive during recovery I have the backup, that's why I do it.
Let's assume that raidz1 will work as expected. Then it will be easier and more logical to recover the system by replacing a disk than by copying everything from the backup.

I'm not trying to convince you of anything, I'm just responding to your comment.

Display More

In the choice between raidz1 and raidz2 the latter is regraded as a "safer" choice - it has higher redundancy. But whatever you choose, if your main pool fails, then would you rather rely on a backup on a single device or another redundant pool? How safe is your backup? I'm sure you're happy to backup say a pc or laptop to your main redundant OMV pool feeling it is secure, yet the backup of your main pool is treated differently. Of course other criteria come into to play as you have mentioned: cost, speed of recovery, etc. The home users priorities will be different to commercial ones. But a backup is not a backup until procedures have been tested, risked weighed against costs, etc. It's always interesting to see whether people put their resources into their main pool or backup and how things might play out when things go wrong.

chente · Mar 17th 2023

Quote from Krisbee

In the choice between raidz1 and raidz2 the latter is regraded as a "safer" choice - it has higher redundancy. But whatever you choose, if your main pool fails, then would you rather rely on a backup on a single device or another redundant pool? How safe is your backup?...

Well of course. Raidz2 is more secure than Raidz1. Where do you want to go? There is always a "safer" option. Raid10 on ZFS is more secure than Raidz2. And if we go further we can mirror 2 Raid10 on ZFS. It would be hard to lose the data that way. And you can go further. Wait, we can make an identical copy of that server on the other side of the world, in case a hurricane destroys our house. Or better two, and that they work at different hours, to try to avoid malware attacks. And also...

Let's not be paranoid. I guess at NASA they do things like that, but we're not NASA. OMV is aimed at domestic users. Even a Raidz1 seems excessive to me for an average home user.

In fact, I kind of understand why Volker chose BTRFS as the preferred file system on OMV. After all, the average OMV user doesn't usually know what a Raid really is. BTRFS is more than enough in 95% of cases. It offers advantages over EXT4 and is supported by debian without licensing issues. It seems to me a very logical, sensible and plausible decision. The other 5% I suppose that we will look for the most twisted paths. I just hope BTRFS is benign with SSDs, write amplification, iowait, etc, we'll see. In this thread I have read that BTRFS could be standardized at root level. I hope no one thinks of formatting an SDHC card in BTRFS, because even openmediavault-flashmemory won't save it.

I am happy with what I have and I hope you are too.

Peace in the world, love and brotherhood for all humanity.

ryecoaaron · Mar 17th 2023

Quote from chente

I hope no one thinks of formatting an SDHC card in BTRFS,

armbian has images using btrfs for root.

chente · Mar 17th 2023

Quote from ryecoaaron

armbian has images using btrfs for root.

oh really? btrfs is surprising, there are very conflicting information on the internet

Krisbee · Mar 17th 2023

Quote from chente

Well of course. Raidz2 is more secure than Raidz1. Where do you want to go? There is always a "safer" option. Raid10 on ZFS is more secure than Raidz2. And if we go further we can mirror 2 Raid10 on ZFS. It would be hard to lose the data that way. And you can go further. Wait, we can make an identical copy of that server on the other side of the world, in case a hurricane destroys our house. Or better two, and that they work at different hours, to try to avoid malware attacks. And also...

Let's not be paranoid. I guess at NASA they do things like that, but we're not NASA. OMV is aimed at domestic users. Even a Raidz1 seems excessive to me for an average home user.

In fact, I kind of understand why Volker chose BTRFS as the preferred file system on OMV. After all, the average OMV user doesn't usually know what a Raid really is. BTRFS is more than enough in 95% of cases. It offers advantages over EXT4 and is supported by debian without licensing issues. It seems to me a very logical, sensible and plausible decision. The other 5% I suppose that we will look for the most twisted paths. I just hope BTRFS is benign with SSDs, write amplification, iowait, etc, we'll see. In this thread I have read that BTRFS could be standardized at root level. I hope no one thinks of formatting an SDHC card in BTRFS, because even openmediavault-flashmemory won't save it.

I am happy with what I have and I hope you are too.
Peace in the world, love and brotherhood for all humanity.

Display More

chente It was just an exchange of ideas, not a criticism of your set-up. In the days of yore people backuped to disk, and then the backuped the backup to tape. Of course you could go on ad inifinitum, hence the risk assessment and judging what type and where to locate backups.

Write amplification is a fact of life with COW filesystems, zfs offers some mitigation by setting recordsize, but AFIAK BTRFS has no equivalent. I believe there are meant to be BTRFS internal optimization for SSDs. One thing missing from OMV at the moment is flexible mount options for BRTFS subvols.

chente · Mar 17th 2023

Quote from Krisbee

It was just an exchange of ideas, not a criticism of your set-up.

I know, I just gave it as an example of my motivation to find out more about BTRFS. I have no intention of modifying it. I don't want to increase my level of paranoia. If with everything I've already done I still lose my files I'll cry and move on with my life

Quote from Krisbee

I believe there are meant to be BTRFS internal optimization for SSDs.

I hope so. There is information on the internet on this subject that makes neurons move. For example https://arxiv.org/pdf/1707.08514.pdf

The problem is to know which is the outdated information and which is really applicable. As an example that document that is from 2017.

Krisbee · Mar 17th 2023

Quote from chente

I hope so. There is information on the internet on this subject that makes neurons move. For example https://arxiv.org/pdf/1707.08514.pdf
The problem is to know which is the outdated information and which is really applicable. As an example that document that is from 2017.

Finding up to date info is a general problem with certain topics in Linux, not least BTRFS. Virtual machines, databases and the number of snapshots seem always to be linked to discussions about write amplification on COW filesystems. I'd like to think things have improved since this interchange:

write amplification, was: very slow "btrfs dev delete" 3x6Tb, 7Tb of data - Chris Murphy

But I have not used BTRFS enough to make informed comment. The official line is here:

btrfs-man5(5) — BTRFS documentation

ryecoaaron · Mar 18th 2023

Quote from Krisbee

In the days of yore people backuped to disk, and then the backuped the backup to tape.

We still do both at work.

crashtest · Mar 18th 2023

Quote from ryecoaaron

We still do both at work.

Tape? Really? The last time I saw tape in use was with the old DLT drives and cartridges.

ryecoaaron · Mar 19th 2023

Quote from crashtest

Tape? Really? The last time I saw tape in use was with the old DLT drives and cartridges.

We have a 30+ drive LTO-9 (45 TB compressed) tape robotic library that backups are written to every weekend (60+ tapes each time last I heard). This is done so the tapes can be stored offsite in underground, climate controlled environment.

crashtest · Mar 19th 2023

Quote from ryecoaaron

We have a 30+ drive LTO-9 (45 TB compressed) tape robotic library that backups are written to every weekend (60+ tapes each time last I heard). This is done so the tapes can be stored offsite in underground, climate controlled environment.

We learn something new everyday. Sounds expensive (45 TB). I do get the offsite, underground and climate controlled condition's. As a backup of last resort, "safe" is required.

ryecoaaron · Mar 19th 2023

Quote from crashtest

Sounds expensive

Our backup infrastructure costs millions per year.

BernH · Mar 19th 2023

Quote from ryecoaaron

We have a 30+ drive LTO-9 (45 TB compressed) tape robotic library that backups are written to every weekend (60+ tapes each time last I heard). This is done so the tapes can be stored offsite in underground, climate controlled environment.

We are just about to put in an LTO-9 library ourselves. Only 4 drives but 80 tapes running a daily backup with filled tapes stored off-site. Hopefully it will be more efficient that our current single LTO-6 manual drive.

ryecoaaron · Mar 19th 2023

Quote from BernH

We are just about to put in an LTO-9 library ourselves. Only 4 drives but 80 tapes running a daily backup with filled tapes stored off-site. Hopefully it will be more efficient that our current single LTO-6 manual drive.

Between the LTO-9 being quite a bit faster and having four, I think you will be happy.

DollaBill · Apr 27th 2023

Quote from Krisbee

Obviously this means resorting to the CLI and making use of snapper (with snap-sync?) and /or btrbk.

Do these do anything essential? I just use BTRFS on my storage drives and I'm trying to figure out if there's anything I need to be wary about combining with Snapraid.

The old Snapraid FAQ mentioned needing the libblkid library (which ryecoaaron was kind enough to add to the next test build), and that "multiple Btrfs snapshots are not supported". Is this a solved problem? Do I need to use a 3rd party script if I'm not interested in running the Snapraid sync off of BTRFS snapshots? I'm assuming Snapraid doesn't have low level access to the drive, but I want to make sure that adding an exclusion for the .snapshots subvolume is enough for it to play nice with the new features votdev mentioned.

angelogia · Dec 6th 2023

Just installed OMV6 6.9.9-1 on bare metal with 500GB SSD and 3 1TB spinning disks. Create a software RAID5 array and i'm at the create filesystem stage. I click on the + and select BTRFS but when I click on the Profile drop down RAID5 is not there. Am I supposed to select "Single" ? The software RAID5 array is there under Devices (and I selected it)

Thank you for your help

votdev · Dec 6th 2023

Quote from angelogia

Just installed OMV6 6.9.9-1 on bare metal with 500GB SSD and 3 1TB spinning disks. Create a software RAID5 array and i'm at the create filesystem stage. I click on the + and select BTRFS but when I click on the Profile drop down RAID5 is not there. Am I supposed to select "Single" ? The software RAID5 array is there under Devices (and I selected it)

Thank you for your help

A software RAID is using mdadm. If you want to use BTRFS on top of that you need to use Single in the drop down. What you can select in the BTRFS file system creation dialog is the RAID mode that is done by BTRFS itself. RAID5 seems to be not stable at the moment so this is not supported via the UI. You can still create a BTRFS RAID5 via the CLI and mount the file system afterwards via UI.

Participate now!