mergerfs questions

  • suppose I have /dir1 shared on one drive, and /dir2 shared on another. Can i use mergerfs to create a readonly combined share? (For write purposes I would write files to the drive that I wanted to directly through the discrete shares.) Does the mergerfs automatically stay updated if I upload files to dir1 or dir2? How much space does the mergerfs share directory need? Should I create it on the system drive or if on the same drive as dir1 or dir2, should it be excluded from the snapraid array?


    Thanks

  • short answer = NO.


    Long Answer = You need to rename /dir2 on second drive as /dir1 (now you have /dir1 on one drive and /dir1 on second drive), now you can MergerFS to have one big /dir1 that contains all files in /dir1 of both disk.

  • (For write purposes I would write files to the drive that I wanted to directly through the discrete shares.) Does the mergerfs automatically stay updated if I upload files to dir1 or dir2?



    Thanks

    Yes, you can write or read files directly from one drive or the other driver and /dir1 must be stay updated because that is the pourpouse of a mergerFS

  • short answer = NO.


    Long Answer = You need to rename /dir2 on second drive as /dir1 (now you have /dir1 on one drive and /dir1 on second drive), now you can MergerFS to have one big /dir1 that contains all files in /dir1 of both disk.

    Ok that makes sense I suppose. But can I have two shares both named dir1 for write purposes? I suppose I could put an empty directory or file in each one to denote which drive it was on if I had 2 shares with the same name.

  • you can have 2 directories on two disk with same name, and then you can share it with diferent name


    eg:


    /hd0/folder1 -> Share it as Dir1
    /hd1/folder1 -> Share it as Dir2


    and finally a mergerFS share named Folder1 that merge hd0 & hd1 folders

  • how do you do mergerfs on the folder level? I installed the union filesystems plugin and I only see the option to use whole drives as the branches.


    Thanks

    That seems to be a limitation of the plugin. Using whole drives as branches isn't suitable for me either.


    The way I did it is to manually edit /etc/fstab and create the mergerfs mountpoints outside of the [openmediavault] section.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    • Offizieller Beitrag

    The aufs and mhddfs plugins used to use shared folders but it was hard coded amount. The unionfilesystems plugin could be modified to use those but it would make the plugin harder to use and be a big code change.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Can you paste your fstab so I can see what that looks like?

    # >>> [sftp-mergerfs]


    /media/41991950-4d12-4475-86b8-ba54ec09323b/multimedia-content-d1/movies:/media/a6e6252d-5a8f-4e9b-88b3-46bef35b01a0/multimedia-content-d2/movies:/media/1237e230-7f7b-4d27-9a48-0ec0d7fd6d4e/multimedia-content-d3/movies:/media/01e83c5a-5688-4963-b671-0c7ee6d4652d/multimedia-content-d4/movies /media/41991950-4d12-4475-86b8-ba54ec09323b/sftp/outgoing/movies fuse.mergerfs defaults,nonempty,allow_other,use_ino,fsname=mergerfs-movies 0 0


    /media/41991950-4d12-4475-86b8-ba54ec09323b/multimedia-content-d1/music:/media/a6e6252d-5a8f-4e9b-88b3-46bef35b01a0/multimedia-content-d2/music:/media/1237e230-7f7b-4d27-9a48-0ec0d7fd6d4e/multimedia-content-d3/music:/media/01e83c5a-5688-4963-b671-0c7ee6d4652d/multimedia-content-d4/music /media/41991950-4d12-4475-86b8-ba54ec09323b/sftp/outgoing/music fuse.mergerfs defaults,nonempty,allow_other,use_ino,fsname=mergerfs-music 0 0


    /media/41991950-4d12-4475-86b8-ba54ec09323b/multimedia-content-d1/tv-series:/media/a6e6252d-5a8f-4e9b-88b3-46bef35b01a0/multimedia-content-d2/tv-series:/media/1237e230-7f7b-4d27-9a48-0ec0d7fd6d4e/multimedia-content-d3/tv-series:/media/01e83c5a-5688-4963-b671-0c7ee6d4652d/multimedia-content-d4/tv-series /media/41991950-4d12-4475-86b8-ba54ec09323b/sftp/outgoing/tv-series fuse.mergerfs defaults,nonempty,allow_other,use_ino,fsname=mergerfs-tv-series 0 0


    /media/41991950-4d12-4475-86b8-ba54ec09323b/multimedia-content-d1/ufc-ppv:/media/a6e6252d-5a8f-4e9b-88b3-46bef35b01a0/multimedia-content-d2/ufc-ppv:/media/1237e230-7f7b-4d27-9a48-0ec0d7fd6d4e/multimedia-content-d3/ufc-ppv:/media/01e83c5a-5688-4963-b671-0c7ee6d4652d/multimedia-content-d4/ufc-ppv /media/41991950-4d12-4475-86b8-ba54ec09323b/sftp/outgoing/ufc-ppv fuse.mergerfs defaults,nonempty,allow_other,use_ino,fsname=mergerfs-ufc-ppv 0 0


    # <<< [sftp-mergerfs]



    Those are four very long lines. They appear wrapped here but aren't in the file.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

  • Thanks. So if I'm reading this right, you have 4 drives with the same 4 folders and the 4 folders are mounted on the first drive in the outgoing folder using mergerfs? I'm not sure what the nonempty option is - can't find any info on that.


    If I wanted to make my mergerfs read only, should I mount it normally and make it read only at the share level? Or should I add category.create=erofs on the fstab line?


    Thanks

  • Thanks. So if I'm reading this right, you have 4 drives with the same 4 folders and the 4 folders are mounted on the first drive in the outgoing folder using mergerfs? I'm not sure what the nonempty option is - can't find any info on that.


    If I wanted to make my mergerfs read only, should I mount it normally and make it read only at the share level? Or should I add category.create=erofs on the fstab line?


    Thanks


    Yes to the first question. Not sure how nonempty got in there, maybe left over from when I was running AUFS. I have removed nonempty. Thanks for pointing that out.


    I set the permissions I want on the mountpoint directories themselves, not in an fstab entry. I can't have it all read only because I have to add new files into the drives from time to time.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

  • I was thinking I'd have the mergerfs be read only and just use that share to make it easier to browse and play files from my Kodi box. I would still have individual r/w shares for the branches that I would use to put files on the server. Does that make sense? I suppose it doesn't matter if I set the mergerfs read only since the Kodi user won't have write permissions anyway. What I'm wondering is if setting the mergerfs to read only increases performance...


  • Yes to the first question. Not sure how nonempty got in there, maybe left over from when I was running AUFS. I have removed nonempty. Thanks for pointing that out.


    I set the permissions I want on the mountpoint directories themselves, not in an fstab entry. I can't have it all read only because I have to add new files into the drives from time to time.


    I got mergerfs working for me and I figured out why you had 'nonempty' in there. It's a fuse option.


    Zitat


    nonempty
    Allows mounts over a non-empty file or directory. By default these mounts are rejected to prevent accidental covering up of data, which could for example prevent automatic backup.


    I ended up adding the nonempty option to mine after I edited fstab and did 'mount -a' and fuse complained.


    After some non-scientific speed testing, it appears that write performance through mergerfs is about 50% vs. writing directly to a branch. Read performance is about 75%.

    • Offizieller Beitrag

    it appears that write performance through mergerfs is about 50% vs. writing directly to a branch. Read performance is about 75%.

    The direct_io flag will change that. If you have direct_io, writes will be faster. If you don't, reads will be faster.

    omv 7.0.4-2 sandworm | 64 bit | 6.5 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.10 | compose 7.1.2 | k8s 7.0-6 | cputemp 7.0 | mergerfs 7.0.3


    omv-extras.org plugins source code and issue tracker - github


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • Thanks. I'm not using the "direct_io" option. For my configuration, I was thinking I would want reads to be faster for streaming to Kodi. But on second thought, maybe I don't need that much performance for streaming videos and I should enable direct_io to make it more convenient to write files to the server.

  • I wanted to come back here to say that I enabled "direct_io" and only saw a minor increase in performance. Then, instead of browsing the merger-fs share using gvfs on my client machine (Archlinux), I mounted it using /etc/fstab just like my other shares, and now the performance is basically indistinguishable. I've never been able to max out a gigabit connection using gvfs, it just seems slow to me. :(


    So bottom line for me is that merger-fs is awesome, gvfs sucks.

  • I realized one of my questions from the OP never was addressed. That is, if I add the mergerfs mount point on a drive that is part of the snapraid pool, do I need to exclude that point in the snapraid configuration to avoid issues? Or is snapraid smart enough to figure out what's going on?


    Thanks

  • Is there some reason you want to have both a snapraid pool and a mergerfs pool?

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!