mergerfs.balance doesn't work

  • The balance tool does not care about existing paths. If you care about path preservation then you shouldn't use mergerfs.balance. That said... by using the exclude path option you could exclude specific drives in the pool from being targets so you could balance files to a subset of drives. It might not be too much work to add a flag to filter targets by available directory but I'm not sure it'd work too well TBH.


    It'd be cool to make a tool which would try to keep data locality while also balancing across drives but it'd be a decent amount more work than what I've done so far with a limited user base.

  • @ryecoaaron - should we split this thread from post #3 and title it "exclude dirs - Feature Request - MergerFS" ?


    probably more would be interested in finding this little nugget but won't see it hidden in a "doesn't work" thread. (and yes I know I kindda skewed the post)

    • Offizieller Beitrag

    should we split this thread from post #3 and title it "exclude dirs - Feature Request - MergerFS" ?

    The whole thread seems to be about the mergerfs balance utility. I don't see much reason to split it.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • @trapexit, I am getting an error that I do not know what to do with.
    Sure would appreciate your guidance.
    Thank you


    Here is my input/output:


    root@OMVi3:/# python --version
    Python 3.5.3
    root@OMVi3:/# mergerfs.balance -s 10M -e "*aquota*" -e "snapraid*" -e "stignore*" -E "*docker/*" -E "*stfolder/*" /srv/7911fa44-a322-442c-a434-6c2ad2f3b1ff/
    /usr/local/bin/mergerfs.balance: line 7: syntax error near unexpected token `newline'
    /usr/local/bin/mergerfs.balance: line 7: `<!DOCTYPE html>'



    How I installed:


    root@OMVi3:/# cd /usr/local/bin/
    root@OMVi3:/# wget https://github.com/trapexit/me…4c87/src/mergerfs.balance
    Then made executable.

    Version7.0-32 (Sandworm)
    ProcessorAMD EPYC 7302P 16-Core Processor
    KernelLinux 6.1.15-1-pve
    HardwareDell R7515
  • …by using the exclude path option you could exclude specific drives in the pool from being targets so you could balance files to a subset of drives. It might not be too much work to add a flag to filter targets by available directory but I'm not sure it'd work too well TBH.


    I am appreciating mergerfs.balance. I recently brought a drive online that is 150% the capacity of the next largest drive. It is great to have this tool to redistribute and unburden my smaller drives.


    Smaller/Older drives brings me to my query.
    Is there a way to use mergerfs.balance to empty a drive to prepare it for removal of service?


    Thank you
    Jerry

    Version7.0-32 (Sandworm)
    ProcessorAMD EPYC 7302P 16-Core Processor
    KernelLinux 6.1.15-1-pve
    HardwareDell R7515
  • Is there a way to use mergerfs.balance to empty a drive to prepare it for removal of service?

    Is there a reason you cannot remove the drive you wish to empty from the defined mergerfs pool and then copy or move its contents to a location in the pool?

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

  • Is there a reason you cannot remove the drive you wish to empty from the defined mergerfs pool and then copy or move its contents to a location in the pool?

    Nope, yet that is not the point of my query. It would be nice to be able to remove the drive's content without taking the server out of service.

    Version7.0-32 (Sandworm)
    ProcessorAMD EPYC 7302P 16-Core Processor
    KernelLinux 6.1.15-1-pve
    HardwareDell R7515
  • Where did I say to take the server out of service or even physically touch the drive?

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    Einmal editiert, zuletzt von gderf ()

  • Is there a reason you cannot remove the drive you wish to empty from the defined mergerfs pool and then…

    I assumed that at the time the drive is removed from the pool its data too would be unavailable. If so each service that depends on the data would need to be downed. My question was not asking how to remove a drive from the pool and copy or move its contents. Rather I was checking if mergerfs.balance or any of the mergerfs-tools would have the ability to automate the process.


    Jerry

    Version7.0-32 (Sandworm)
    ProcessorAMD EPYC 7302P 16-Core Processor
    KernelLinux 6.1.15-1-pve
    HardwareDell R7515
    • Offizieller Beitrag

    I assumed that at the time the drive is removed from the pool its data too would be unavailable. If so each service that depends on the data would need to be downed. My question was not asking how to remove a drive from the pool and copy or move its contents. Rather I was checking if mergerfs.balance or any of the mergerfs-tools would have the ability to automate the process.

    How often do you remove a drive? I would think you could go a different direction...
    1 - add new drive
    2 - copy all data from drive to be removed to new drive
    3 - add new drive and remove old drive from pool at same time
    4 - optional run balance


    I know this doesn't work if you don't have a free sata port but it is much easier especially for something that doesn't happen often.

    omv 7.0.5-1 sandworm | 64 bit | 6.8 proxmox kernel

    plugins :: omvextrasorg 7.0 | kvm 7.0.13 | compose 7.1.4 | k8s 7.1.0-3 | cputemp 7.0.1 | mergerfs 7.0.4


    omv-extras.org plugins source code and issue tracker - github - changelogs


    Please try ctrl-shift-R and read this before posting a question.

    Please put your OMV system details in your signature.
    Please don't PM for support... Too many PMs!

  • I assumed that at the time the drive is removed from the pool its data too would be unavailable. If so each service that depends on the data would need to be downed. My question was not asking how to remove a drive from the pool and copy or move its contents. Rather I was checking if mergerfs.balance or any of the mergerfs-tools would have the ability to automate the process.
    Jerry

    That's an incorrect assumption. The data is always located on a physical disk regardless if it is or isn't part of a mergerfs pool, and that data is always available via the drive's mount point. In any case, if you want to move the data from one drive to another, or others in the case of a mergerfs pool, then there has to enough room available on the destination drive or drives.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!