UnionFS: No space left on device but has 2.5TB free

    • OMV 4.x

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • UnionFS: No space left on device but has 2.5TB free

      Hi,
      I'm having this annoying problem that for random folders on my share I got the following error when creating files: No space left on device
      My unionFS disk (mounted on /srv/ ) has over 2.5 TB left. The strange thing is this happens for random folders.
      Eg:

      /Movies/USA : no problem writing files
      /Movies/EUR : no space left on device

      What could be the problem please?

      kr
      wouter
    • I'm assuming that you're having the same issue, outlined above, as well.
      _________________________________________________

      Here's what (most likely) happened:
      With "Existing Path, most free space" - when you set up a shared folder, the process establishes a folder (or a path) at the root of the merged drive. "Existing path" sets up a single folder on a single drive, in the merged UnionFS drive. (If a second folder is created, we'll say documents as an example, it would go on another drive but that's not relevant to this issue.)

      We'll say the folder you set up is "Videos" and it stores video files (very large). Since the first directive is "existing path", the second directive "most free space" is ignored because the "Videos" path exists. This means every time a video file is saved to the Videos folder, it goes to that single folder on one drive. If you add a lot of content, the drive fills up rapidly.
      _________________________________________________

      On the fix, you're halfway there. But note that the "most free space" policy is only effective from the time it's implemented. UnionFS will not re-balance your storage for you.

      Using the "Videos" folder as an example:
      You'll have to move some of the content under; /dev/sdd1/Videos/ to /dev/sdc1/Videos/ and - or /dev/sdb1/Videos/ , etc.
      (If the Videos folder doesn't exist on the destination drive, create it but make sure the name exactly matches the folder name as it is on /dev/sdd1, cap's and all.)

      You don't have to completely balance files among all drives. You just need to move enough to take your filled drive down to where it has 10 or 15% free space. Thereafter, the "most free space" policy will direct future content to drives with more space.

      _________________________________________________

      Since "most free space" distributes content among all drives in the Union, you might want to give some thought to dedicating 1 of those 4 drives to SNAPRAID.
    • @flmaxey

      Yeah I read that aswell somewhere, is it possible to remove that file?

      "You'll have to move some of the content under; /dev/sdd1/Videos/ to /dev/sdc1/Videos/ and - or /dev/sdb1/Videos/ , etc."
      I removed some files couple of days ago, and it still telling me no space left now when the drive filled up again, I will try again..
      How does one move easily from one drive to another when using unionFS?
    • bowe wrote:

      I removed some files couple of days ago, and it still telling me no space left now when the drive filled up again, I will try again..
      You'll have to move more, maybe 100's of Gb more. Get it under 10 to 15%. Once the drive is green again, in the GUI, you'll be OK.

      bowe wrote:

      How does one move easily from one drive to another when using unionFS?
      The fastest way to move files is with Midnight Commander. See page 57 of this guide (the current version) for how to install MC, what it is and how it works. You could use WinSCP (how to set WinSCP up is in the guide as well) but it uses a temporary folder which is slow.
    • If you move the files to another drive, inside the drive Union and into an identical folder as noted above, you're done. The new policy will start filling the other drives in the union, with files as you add them, going forward.

      bowe wrote:

      around 200GB, then just move it back
      I'm not sure what you mean by, "just move it back". You'd be filling up the already too full drive again. UnionFS doesn't care which drive the files are on as long as the folder structure is identical. ((**Also note that you DO NOT want two copies of the same files on two different drives, in the same Union.**)

      For example;

      /dev/sdb/Videos/file1
      /dev/sdc/Videos/file2
      /dev/sdd/Videos/file3

      equates to (as you'll see it):

      /Uniondrive/Videos/file1, file2, file3

      Where the files are, on the individual drives, doesn't matter because UnionFS merges them together.
      ___________________________________________________

      With the new policy in place , you could move files out of the NAS, and onto a client, then move them back again. In that case, with the original files on the NAS deleted, when files are moved back the new policy would distribute them to drives with the "most free space".
      In my opinion, this is somewhat risky. Massive file copies, done with Windows explorer is prone to errors. If you do it this way, move small chunks of 5 or 10GB at a time.

      The post was edited 1 time, last by crashtest ().

    • @flmaxey

      Im not sure I understand this issue...

      This is my setup, sharedfolder/media/incoming/all fetched
      which is moved upon completion to, sharedfolder/media/completed

      I did set up a new sharedfolder just for this case. sharedfolder/test/ and moved 700gb ish to shared so my unionFS filesystem would be "green" then raised min allowed disk to 415GB and restarted OMV (just to test) Then tried to copy sharedfolder/test back to sharedfolder/media/completed and got the same message "No space left"


      gderf wrote:

      Have you seen the FAQ entry on github.com/trapexit/mergerfs ?

      Why do I get an "out of space" / "no space left on device" / ENOSPC error even though there appears to be lots of space available?
      This is my options in UnionFS defaults,allow_other,use_ino,moveonenospc=true Are those not correct?
    • bowe wrote:

      This is my setup, sharedfolder/media/incoming/all fetched
      which is moved upon completion to, sharedfolder/media/completed
      I don't want to add any confusion as this is way I want to go, but if I have read @flmaxey explanation correctly the issue is the policy;

      Existing path -> Video that is why one drive is filling up, irrespective of any sub folders, what he has suggested is to change the policy to Most free space, move 200GB+ off of the UnionFS to an external drive, that's move not copy. Then move it back to UnionFS and it should sort itself out.
      Raid is not a backup! Would you go skydiving without a parachute?
    • moving a file back to /srv/123123-123-123123/media from /sharedfolders/test

      and it's super slow, tried smaller files aswell but it's the same speed. can't get over 1mb/s when transfering.
      I've no idea why, someone experienced this also?
      Images
      • omv3.png

        37.75 kB, 1,658×210, viewed 40 times