How to force UnionFS to move data in accordence with new free space policy?

    • OMV 4.x
    • How to force UnionFS to move data in accordence with new free space policy?

      When I initially created my UnionFS pool I didn't set the free space value to be large enough (left it at default, now set to 5% of max disk size) and at this point one of my data disks is completely full which I think is causing some issues with SnapRAID's sync attempts. So I'd like to know what the best way is to get the files moved from the full disk to the other data disks in the pool, obeying the new free space rule I set.
    • I'd use mc to do the move (Midnight Commander).

      On the CLI:
      apt-get install mc

      then
      mc

      If you're that close on one of your drives you might want to consider using a new policy;
      Create Policy: Most Free Space

      (It sounds like you may be using this already.)
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)
    • flmaxey wrote:

      I'd use mc to do the move (Midnight Commander).

      On the CLI:
      apt-get install mc

      then
      mc

      If you're that close on one of your drives you might want to consider using a new policy;
      Create Policy: Most Free Space

      (It sounds like you may be using this already.)

      I'm using Existing Folder, Most Free Space. I'll try out mc though, thanks. Although I'm still kind of confused on what I'm supposed to be doing to get this fixed, it sounds like I have to manually move the files? so no automatic balancing like drivepool has?

      The post was edited 1 time, last by JunkKnight ().

    • JunkKnight wrote:

      I'm using Existing Folder, Most Free Space. I'll try out mc though, thanks. Although I'm still kind of confused on what I'm supposed to be doing to get this fixed, it sounds like I have to manually move the files? so no automatic balancing like drivepool has?
      "No". The answer to automatic balancing is "No". :) The policy you choose doesn't work that way.

      For example; "Existing Folder - Most Free Space" would do the following if you copied the folder "Videos" onto it:
      1. UnionFS would select the drive with the most free space and put the Videos folder and the folders' files on it.
      We'll say that's drive one.
      2. The next new folder "Documents" would go to drive 2 since, at this point, drive 2 would have more free space.
      3. The kicker is, when using "Existing Folder - Most Free Space", the Videos folder already exists on drive one. All new additions to the existing folder "Videos", will always go to drive one. If you had a lot of videos to start, and added more over time, drive 1 could easily go to the limit.

      If you would have used "Most Free Space", the directory structure on both drives will be, more or less, the same and each new file added would go to the drive with the most free space. In this manner, the policy "Most Free Space" balances storage over the pool of UnionFS drives.
      ________________________________________________________

      But, @gderf is 100% correct. Manual moves are now required to balance your storage. At this point you'll need to move root folders (I'd do one at at time) from the drive that's nearly full to the drive with some room on it. (And since it's probably the cause of this problem; I'd leave Videos or whatever you named it where it is, and move something else.) Get the two drives close to the same capacity.

      Thereafter, use the policy "Most Free Space", and balancing should occur on it's own.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)

      The post was edited 1 time, last by flmaxey: edite ().

    • Policies are used at the point the request is made. mergerfs doesn't actively move data around and that's very purposeful. Best to leave such complicated tasks to the user. If your drives are filling in ways you don't want its likely you chose the wrong policy. But tools like mergerfs.balance can help but should only be used if you don't care where data goes as I never made its algo any more complicated than move files from the fullest drives to the least fullest.
    • trapexit wrote:

      Policies are used at the point the request is made. mergerfs doesn't actively move data around and that's very purposeful. Best to leave such complicated tasks to the user. If your drives are filling in ways you don't want its likely you chose the wrong policy. But tools like mergerfs.balance can help but should only be used if you don't care where data goes as I never made its algo any more complicated than move files from the fullest drives to the least fullest.
      That's a pretty slick tool you created. I'd like to see it become part of Unionfs/mergerfs in the GUI, so it's a bit more friendly for beginners.
      Good backup takes the "drama" out of computing
      ____________________________________
      Primary: OMV 3.0.99, ThinkServer TS140, 12GB ECC, 32GB USB boot, 4TB+4TB zmirror, 3TB client backup.
      Backup: OMV 4.1.9, Acer RC-111, 4GB, 32GB USB boot, 3TB+3TB zmirror, 4TB Rsync'ed disk
      2nd Data Backup: OMV 3.0.99, R-PI 2B, 16GB boot, 4TB WD USB MyPassport - direct connect (no hub)
    • Users Online 1

      1 Guest