How to force UnionFS to move data in accordence with new free space policy?

  • When I initially created my UnionFS pool I didn't set the free space value to be large enough (left it at default, now set to 5% of max disk size) and at this point one of my data disks is completely full which I think is causing some issues with SnapRAID's sync attempts. So I'd like to know what the best way is to get the files moved from the full disk to the other data disks in the pool, obeying the new free space rule I set.

  • Don't move the files to the ufs mountpoint. Move them to the actual physical disk mountpoints instead.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    • Offizieller Beitrag

    I'd use mc to do the move (Midnight Commander).


    On the CLI:
    apt-get install mc


    then
    mc

    If you're that close on one of your drives you might want to consider using a new policy;
    Create Policy: Most Free Space

    (It sounds like you may be using this already.)

  • I'm using Existing Folder, Most Free Space. I'll try out mc though, thanks. Although I'm still kind of confused on what I'm supposed to be doing to get this fixed, it sounds like I have to manually move the files? so no automatic balancing like drivepool has?

  • You already know which disk is 100% full, right? Just move some files from that actual disk to another actual disk - FORGET ABOUT THE UFS POOL at this point.



    mergerfs.balance from the mergerfs tools can do this for you, see:


    https://github.com/trapexit/mergerfs-tools


    But trust me, just do it manually this time.

    --
    Google is your friend and Bob's your uncle!


    OMV AMD64 7.x on headless Chenbro NR12000 1U 1x 8m Quad Core E3-1220 3.1GHz 32GB ECC RAM.

    • Offizieller Beitrag

    I'm using Existing Folder, Most Free Space. I'll try out mc though, thanks. Although I'm still kind of confused on what I'm supposed to be doing to get this fixed, it sounds like I have to manually move the files? so no automatic balancing like drivepool has?

    "No". The answer to automatic balancing is "No". :) The policy you choose doesn't work that way.


    For example; "Existing Folder - Most Free Space" would do the following if you copied the folder "Videos" onto it:
    1. UnionFS would select the drive with the most free space and put the Videos folder and the folders' files on it.
    We'll say that's drive one.
    2. The next new folder "Documents" would go to drive 2 since, at this point, drive 2 would have more free space.
    3. The kicker is, when using "Existing Folder - Most Free Space", the Videos folder already exists on drive one. All new additions to the existing folder "Videos", will always go to drive one. If you had a lot of videos to start, and added more over time, drive 1 could easily go to the limit.


    If you would have used "Most Free Space", the directory structure on both drives will be, more or less, the same and each new file added would go to the drive with the most free space. In this manner, the policy "Most Free Space" balances storage over the pool of UnionFS drives.
    ________________________________________________________


    But, @gderf is 100% correct. Manual moves are now required to balance your storage. At this point you'll need to move root folders (I'd do one at at time) from the drive that's nearly full to the drive with some room on it. (And since it's probably the cause of this problem; I'd leave Videos or whatever you named it where it is, and move something else.) Get the two drives close to the same capacity.


    Thereafter, use the policy "Most Free Space", and balancing should occur on it's own.

  • Policies are used at the point the request is made. mergerfs doesn't actively move data around and that's very purposeful. Best to leave such complicated tasks to the user. If your drives are filling in ways you don't want its likely you chose the wrong policy. But tools like mergerfs.balance can help but should only be used if you don't care where data goes as I never made its algo any more complicated than move files from the fullest drives to the least fullest.

    • Offizieller Beitrag

    Policies are used at the point the request is made. mergerfs doesn't actively move data around and that's very purposeful. Best to leave such complicated tasks to the user. If your drives are filling in ways you don't want its likely you chose the wrong policy. But tools like mergerfs.balance can help but should only be used if you don't care where data goes as I never made its algo any more complicated than move files from the fullest drives to the least fullest.

    That's a pretty slick tool you created. I'd like to see it become part of Unionfs/mergerfs in the GUI, so it's a bit more friendly for beginners.

  • Thanks. I've been wanting to rewrite it to have a nicer workflow. Currently it just starts moving files around. Preferably it'd calculate what should be moved and where and then provide a summary before acting on it. But... it works, I don't use it that often, and I've been working on other things.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!