Can we just kill this discussion
Feel free to close the thread if you think it helps...it wont change the situation though.
Is Snapraid compatable with new BTRFS Snapshots?
-
- OMV 6.x
- DollaBill
-
-
MarcS Your position does not hold. It doesn't make sense to use Snapraid with BTRFS. BTRFS replaces SnapRaid. Therefore the way in which BTRFS has been implemented in OMV is irrelevant.
If, as I read a while ago, BTRFS ends up being the standard file system on OMV, SnapRaid will most likely be deprecated and in that case there would be no point in continuing to maintain the plugin.
Would you also use SnapRaid with ZFS? Don't answer this question please, it was a rhetorical question
-
There is a s no need for a knob to switch on/off sibvolumes. I already posted a workaround/solution to use simple directories instead of creating a new subvolume.
-
It is causing problems for the servers based on Snapraid.
No it isn't. snapraid just doesn't calculate parity IF you have subvolumes. Since most people don't use snapraid with btrfs and even less have subvolumes, I don't see this as a problem. There may be people who actually want snapraid to ignore subvolumes. Volker has given a workaround. We don't have to close this thread but I feel you are beating a dead horse. You should ask the snapraid author to support subvolumes since this is such a problem.
-
If the directory already exists, the shared folder creation mechanism does not create a new subvolume if you point to this directory in the shared folder creation dialog in the UI. The UI gives you more or less already that hint.
I mean it's a bit of a clue you can follow, but seems like a stretch to go from a button being greyed out to 'some shares are subvolumes, some aren't', or how you might have ended up with both. Ideally there would be some more direct indication of what's going on.
sharedfolders aren't used by snapraid at all. btrfs already does much of what snapraid does. Why use both? https://www.snapraid.it/compare
Mainly because RAID setups are considered to be one of the less stable features in BTRFS, and where you're most likely to encounter problems with it. Which could be a significant issue if we're still considering the imminent demise of all other filesystems in OMV. It's also nice that you don't have to stripe data or rewrite your existing disks when adding them to snapraid, which can be a saving grace come resilvering time. BTRFS for me was just a way of tacking on multiple snapshots, and instant hardlinked copies via COW.
If, as I read a while ago, BTRFS ends up being the standard file system on OMV, SnapRaid will most likely be deprecated and in that case there would be no point in continuing to maintain the plugin
I know the plugins have always been somewhat second-class citizens, but I think a lot of the community really depend on snapraid as part of their setup, and it's a real bummer to hear it might just evaporate as an option. That's a pretty significant migration project I don't think a lot of us were counting on. Maybe the snapraid plugin could just include a note that you need to pre-generate any shares you want it to include?
-
it's a real bummer to hear it might just evaporate as an option.
Where are you getting that? Just because it doesn't work with btrfs snapshot? Who said you won't be able to continue to use snapraid without support for snapshot? I can't help that snapraid doesn't support subvolumes. Feel free to ask the snapraid author to support them.
I know the plugins have always been somewhat second-class citizens
In what way?
Mainly because RAID setups are considered to be one of the less stable features in BTRFS
Only raid 5/6.
Which could be a significant issue if we're still considering the imminent demise of all other filesystems in OMV.
I have said before that I don't ever want ext4/xfs support to go away even if I have to write a plugin to continue their support.
instant hardlinked copies via COW
A true hard link is instant on every filesystem
Maybe the snapraid plugin could just include a note that you need to pre-generate any shares you want it to include?
It should be added to the wiki.
-
Just meaning that anything in community plugins isn't guaranteed to be available in the future, and usually doesn't get much mention in the main documentation. Not that I'm not grateful. I can't imagine the amount of work that goes into maintaining them, let alone doing writeups and answering endless questions.
I know BTRFS has seen a lot of improvement the last few years, but I think 2+1 drives is a pretty common setup and it seems like that still has a ways to go. Anecdotally I've heard complaints about its RAID reliability in general, but there could be some confirmation bias there. I'd also like to avoid a performance hit as the drives fill.
Who said you won't be able to continue to use snapraid without support for snapshot?
Maybe I misunderstood what Chente was getting at. So snapraid will definitely continue to be supported in the future?
-
Feel free to ask the snapraid author to support them.
I have asked Andrea (Snapraid author) to support Subvolumes and he is not against it. Its just a matter of time to implement the changes and it wont happen quickly he says. After all, BTRFS remains an unstable filesystem and unusable for RAID 5/6 use which in a NAS is probably 90% of the use cases. Nobody is arguing that Snapshots are helpful. I think what we are concerned about is that the way, Snapshots have been implemented in OMV (via Shared Folders) forces everyone into Subvolumes and therefore outof Snapraid, a stable and very usefull NAS application.
-
Just meaning that anything in community plugins isn't guaranteed to be available in the future
You will have to use a product from a very large corporation to guarantee anything.
sually doesn't get much mention in the main documentation
And I don't think it should. I don't expect Volker to maintain omv-extras documentation and we have a very good omv-extras wiki.
So snapraid will definitely continue to be supported in the future?
Since I wrote and maintain the plugin, I will tell you yes.
BTRFS remains an unstable filesystem and unusable for RAID 5/6 use
I really don't want to start this argument again but the first part of that statement is false. I personally have used btrfs for years and never had an issue. And I am quite sure that the folks at OpenSUSE know more about btrfs stability than you or I or 99% of the people saying it is unstable.
-
I think what we are concerned about is that the way, Snapshots have been implemented in OMV (via Shared Folders) forces everyone into Subvolumes and therefore outof Snapraid, a stable and very usefull NAS application.
Use EXT4 or any other file system except BTRFS and you are fine.
-
Since I wrote and maintain the plugin, I will tell you yes.
Awesome. Thanks again for all the work you do. The fact that all of this is available free, let alone open source from volunteer labor is incredible
-
-
Hmm, I am fairly new to OMV and I really like it over the other options in the HomeLab/NAS arena. I want to explain my goals and make sure I am moving in the right direction and have all of the caveats clearly understood.
-I want some form of bitrot protection and drive failure protection (RAID Like)
-I want power efficiency and long drive life from my NAS
-I want efficiency of storage (and funds) from the system
-I want a COW filesystem to work with cheaper SMR drives effectively (lowest priority, WD Blue 8TB is CMR so a cheap option is available)
-I would like the system to be available at anytime
Use case
-I have 4TB of important storage with 1.5TB of that critical storage. This data is backed up externally, and the critical data is also on the cloud and on an offsite external hdd. It is only ever added to and usually only a few times a month.
-I have a huge media library (8tb+ and growing) that has some of the content externally backed up but very little is important.
-BTRFS Raid is basically mirror only so 50% storage efficiency which is to low for cost effectiveness
-ZFS and MDADM+BTRFS would meet my data security needs but both are inflexible and require the whole array to be spinning for any data access. They also restrict you to CMR drives.
-EXT4/XFS plus SnapRaid-MergerFS has only 1 drive spinning for data access (eg watching TV) and is very flexible but the system is not available during Sync operations and although it will work with SMR drives, there is no COW.
-BTRFS plus Snapraid meets all of my goals but requires manual operation and has a lot of caveats.
As I understand it, BTRFS and Snapraid would both provide bitrot detection with Snapraid also providing correction. BTRFS also allows COW and snapshots but in order to use it with Snapraid I would have to create the folder for each share manually and would lose the ability to create a snapshot using the GUI. The current Snapraid plugin is not BTRFS aware and does not use SnapRaid-BTRFS. SnapRaid-BTRFS scripts create a snapshot for Snapraid and then run the sync on the snapshot so the system can remain in use during the snapraid sync.
-So, create each share folder manually and I should have the same operation/level of protection as EXT4+Snapraid-MergerFS but with a COW FS and already migrated to what is clearly the direction OMV is going.
-Manually run/setup Snapraid-BTRFS into cron jobs to attain all goals
-Wait for progress with OMV, BTRFS, and Snapraid integration or learn to code and make a BTRFS-Snapraid plugin for OMV7 (wishful thinking but I may retire in a year or 5)
Am I missing anything or any risks. Is there a better way considering my modest goals. Is there a way to migrate manual shares to a BTRFS Subvolume as support changes?
Thanks in advance.
-
The current Snapraid plugin is not BTRFS aware and does not use SnapRaid-BTRFS. SnapRaid-BTRFS scripts create a snapshot for Snapraid and then run the sync on the snapshot so the system can remain in use during the snapraid sync.
snapraid isn't btrfs aware. The snapraid-btrfs script is a workaround. Supporting this script would be a lot of work since not everyone uses btrfs. I made the large change to make the plugin support multiple arrays and split parity and could barely find anyone to even test it. Doesn't make me want to make another large change for a small handful of users.
make a BTRFS-Snapraid plugin for OMV7
It does not make sense to make this a second plugin.
learn to code
Strangely, those who aren't coders want the most changes...
-
It does not make sense to make this a second plugin.
Strangely, those who aren't coders want the most changes...
No but the exercise of tearing apart your code plus the scripts from BTRFS-Snapraid and Neon's script above might....might just teach me enough to be dangerous with the hope of useful. But, to find the time and resources to re-learn almost everything. Last time I did anything beyond modify a few scripts was in the mid nineties and most of my skills were focused on assembly even then with a touch of VB and C. Yes, I am a dinosaur.
-
to find the time
That is my problem as well. I would have to figure out what the script is doing. When I do that, I would probably want to rewrite it. And that is all before having to make the changes to the plugin. Then getting enough people to test is difficult. Oh, and I don't have hardly any time to do that.
Last time I did anything beyond modify a few scripts was in the mid nineties and most of my skills were focused on assembly even then with a touch of VB and C. Yes, I am a dinosaur.
I probably wrote a quarter of a million lines of VB and a few thousand lines of C in the 90s. No assembly since the early 80s on a Apple II+. I might not be a dinosaur but not a spring chicken either : )
-
No assembly since the early 80s on a Apple II+. I might not be a dinosaur but not a spring chicken either : )
Gee. At 51 that makes me feel young. In the early 80’s I was still in elementary school and junior high, but I was writing basic on my trusty old Vic 20.
-
At 51 that makes me feel young. In the early 80’s I was still in elementary school and junior high, but I was writing basic on my trusty old Vic 20.
I'm 49. I was writing basic programs at 7 on TI-99/4A and Apple II+. Assembly a couple years after that but not a ton.
-
I'm 49. I was writing basic programs at 7 on TI-99/4A and Apple II+. Assembly a couple years after that but not a ton.
Well that lines up about right. I was 9 or 10 when I got the Vic 20 so we actually started around the same time. Aside from a very minor exploration, I never got into assembly, C, VB, etc. I headed into electronics, A/V installation and computerized A/V post production. I did learn a couple of proprietary programming languages for AMX control systems we used to install, and with the bit of basic programming it helped with having the right mind set to learn to program those systems.
When you said you were no spring chicken, I expected older.
-
When you said you were no spring chicken, I expected older.
It was the best term I could come up with that was younger than a dinosaur : )
Jetzt mitmachen!
Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!