unionfilesystems plugin

    • OMV 5.x (beta)
    • unionfilesystems plugin

      In case anyone notices, the latest update of the unionfilesystems plugin will now install omvextras-unionbackend. I separated the plugin from the unionfilesystems backend so that I can create a second plugin that allows you create folder pools like @gderf has wanted forever. To avoid having the change the current unionfilesystems plugin a lot, I decided to make a second plugin. If you want both filesystem pools and folder pools, install both plugins.
      omv 5.3.2 usul | 64 bit | 5.3 proxmox kernel | omvextrasorg 5.2.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Thanks for the update and enabling folder pooling. I installed both plugins:

      openmediavault-unionfilesystems 5.1

      omvextras-unionbackend 5.0

      But where do I find the new folder pooling?
      --
      Google is your friend and Bob's your uncle!

      RAID - Its ability to disappoint is inversely proportional to the user's understanding of it.

      ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.
    • gderf wrote:

      But where do I find the new folder pooling?
      It will be another plugin called openmediavault-mergerfsfolders. But it isn't done yet. Just wanted to explain the necessary change before people started asking why.

      Adoby wrote:

      what is the use-case for folder pools? What is it?
      Some people don't want to pool entire filesystems and want multiple pools of different folders. I don't use it myself but gderf has asked for it for a long time.
      omv 5.3.2 usul | 64 bit | 5.3 proxmox kernel | omvextrasorg 5.2.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Folder pools allow you to easily merge folders on several drives as opposed to drive pools which merge entire drives.
      --
      Google is your friend and Bob's your uncle!

      RAID - Its ability to disappoint is inversely proportional to the user's understanding of it.

      ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.
    • ryecoaaron wrote:

      gderf wrote:

      But where do I find the new folder pooling?
      It will be another plugin called openmediavault-mergerfsfolders. But it isn't done yet. Just wanted to explain the necessary change before people started asking why.Some people don't want to pool entire filesystems and want multiple pools of different folders. I don't use it myself but gderf has asked for it for a long time.
      Thanks for the clarification. Since the openmediavault-mergerfsfolders isn't done yet, may I ask a question? Will it support globbing (* wildcard usage)? I use that like this in fstab:


      Brainfuck Source Code

      1. /srv/dev-disk-by-label-d*/multimedia-content-d*/movies /srv/dev-disk-by-label-d1/sftp/outgoing/movies fuse.mergerfs noforget,..........0 0
      The asterisk allows me to find what I want across every drive and a named subfolder. I never have to change this pool definition when I add new disks so log as I label the disks d1, d2, d3,........dn and the subfolder is present in the root of the new drive.
      --
      Google is your friend and Bob's your uncle!

      RAID - Its ability to disappoint is inversely proportional to the user's understanding of it.

      ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.
    • gderf wrote:

      Will it support globbing (* wildcard usage)?
      It will now. Since I hadn't written that part yet, I can make it support that.
      omv 5.3.2 usul | 64 bit | 5.3 proxmox kernel | omvextrasorg 5.2.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • ryecoaaron wrote:

      gderf wrote:

      Will it support globbing (* wildcard usage)?
      It will now. Since I hadn't written that part yet, I can make it support that.
      Thanks so very much.
      --
      Google is your friend and Bob's your uncle!

      RAID - Its ability to disappoint is inversely proportional to the user's understanding of it.

      ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.
    • So, I have the mergerfsfolders plugin done but I haven't released it. There is an issue when you create a pool with the unionfilesystems plugin and a pool with the mergerfsfolders plugin. The device dropdown in the shared folders can't tell which pool is which. So, it will list duplicates. This is because the first column in /proc/mounts will list 1:2 for more than one pool. Not sure if this is a mergerfs issue, fuse, issue or ? Maybe @trapexit knows?

      If it is a fuse issue, I could make the plugins conflict so that only one is installed at a time. Not sure if people would use both types of pools or not.
      omv 5.3.2 usul | 64 bit | 5.3 proxmox kernel | omvextrasorg 5.2.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • I don't know anything about the plugin but the first column of /proc/mounts is not unique. You can't be relying on that for anything. In mergerfs that is configurable by the fsname option.

      I'm guessing you're having problems because you can't trace back to which mounts are created by which plugin? If that's the case I suspect you'd need to find a way to indicate which is which some other way.
    • trapexit wrote:

      I don't know anything about the plugin but the first column of /proc/mounts is not unique. You can't be relying on that for anything.
      Unfortunately, the plugin can't do anything about it. OMV itself is returning the first column to the plugin.

      trapexit wrote:

      In mergerfs that is configurable by the fsname option.
      That actually works well. I would have to force that option. Maybe I can make the backend work with that.

      trapexit wrote:

      I'm guessing you're having problems because you can't trace back to which mounts are created by which plugin? If that's the case I suspect you'd need to find a way to indicate which is which some other way.
      The plugin can tell which pool belongs to which plugin because of the mount point being stored in the database. But, other parts of OMV's web interface can't tell since they use the first column.
      omv 5.3.2 usul | 64 bit | 5.3 proxmox kernel | omvextrasorg 5.2.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • Looks like this is getting complicated. But no problem if it can't be done as it seems to be a low interest item.

      I haven't changed my hand rolled mounts in years even though I keep adding more drives.
      --
      Google is your friend and Bob's your uncle!

      RAID - Its ability to disappoint is inversely proportional to the user's understanding of it.

      ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.
    • gderf wrote:

      Looks like this is getting complicated. But no problem if it can't be done as it seems to be a low interest item.

      I haven't changed my hand rolled mounts in years even though I keep adding more drives.
      I actually just pushed the plugin out. Adding fsname option of pool_name + ":" + pool_uuid fixed it. Adding the uuid keeps it unique. Adding the pool name first lets people identify it in the output of mount and df. The colon in the middle makes it work with the backend. Let me know what you think.
      omv 5.3.2 usul | 64 bit | 5.3 proxmox kernel | omvextrasorg 5.2.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • trapexit wrote:

      In mergerfs that is configurable by the fsname option.
      Thank you for this suggestion!
      omv 5.3.2 usul | 64 bit | 5.3 proxmox kernel | omvextrasorg 5.2.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!
    • OK, I'll grab a copy and see how it compares to my current mounts.

      Thanks!

      Edit:

      Will this show up as an update or is it a manual install and if so, link?

      Nevermind I found it.
      --
      Google is your friend and Bob's your uncle!

      RAID - Its ability to disappoint is inversely proportional to the user's understanding of it.

      ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.

      The post was edited 1 time, last by gderf ().

    • Folder plugin looks good. I did two tests.

      First one was to just select a few folders. Result was the colon separated string of directories.

      Second test was to use globbing like I do by hand. That worked too. One question though. Will that x-systemd.requires= statement that has * wildcards in it work?

      /srv/dev-disk-by-label-d*/multimedia-content-d*/movies/srv/f75dd4b2-48cc-4cb2-980c-070b82ad80bf fuse.mergerfs defaults,allow_other,cache.files=off,use_ino,category.create=epmfs,minfreespace=4G,fsname=movies-test2:f75dd4b2-48cc-4cb2-980c-070b82ad80bf,x-systemd.requires=/srv/dev-disk-by-label-d*/multimedia-content-d*/movies 0 0
      --
      Google is your friend and Bob's your uncle!

      RAID - Its ability to disappoint is inversely proportional to the user's understanding of it.

      ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.
    • gderf wrote:

      One question though. Will that x-systemd.requires= statement that has * wildcards in it work?
      Shit. I meant to add code to not add that statement when there is a wildcard in it. I will add that.

      ** Edit - 5.0.1 in repo. Thanks for noticing that.
      omv 5.3.2 usul | 64 bit | 5.3 proxmox kernel | omvextrasorg 5.2.4
      omv-extras.org plugins source code and issue tracker - github

      Please read this before posting a question and this and this for docker questions.
      Please don't PM for support... Too many PMs!

      The post was edited 1 time, last by ryecoaaron ().

    • mergerFS folder allows you to select and merge specific folders.

      Union FS only allows you to select entire drives to merge.
      --
      Google is your friend and Bob's your uncle!

      RAID - Its ability to disappoint is inversely proportional to the user's understanding of it.

      ASRock Rack C2550D4I C0 Stepping - 16GB ECC - Silverstone DS380 + Silverstone DS380 DAS Box.