snapraid oom, outofpairity

1. offizieller Beitrag

    I'm crashing snapraid. This is in OMV5. I did test runs on the system with a 1T drive as the pairity and only the 500 and 700g as data & content, and things worked ok.


    I upgraded drives and now have 3 drives for data and content (500Gb, 700Gb, and 3T) and one 3T drive for pairity. Since then, every time I run sync it uses all the memory and crashes oom error, unless I turn on swap, in which case it swaps its ass off and never completes the sync. I just started 'fresh' with block size increased from 256 to 512, deleted pairty & content files, and ran a new sync. Same thing this time, high mem usage and load for hours, but now the the pairity drive filed up completely. I don't get it, as I only have 1.5T of data, mostly music and movies. I shouldn't be using this much memory or pairity space.


    Union1 is simply a mergerfs pool of the 3 data drives, not included in the snapraid config.


    snapraid config from omv interface

    What the heck am I doing wrong?

    Mark D

    OMV6.1.0 upgraded from 5 on a Core2Duo

    mergerfs & snapraid

    Docker for urbackup, omada controller, adguard.

    Updated snapraid to 11.5 from 11.3. Repeated delete pairity file, content files, and rescan. Still ran out of pairity.


    5194 MiB used? I only have 1.4T of data... This thing was supposed to use 1G mem for 16T of data. I'm confused :(


    Log file starts.



    Middle of the of the log file, after scanning the disks, until outofpairity is reached:


    ~~~

    Mark D

    OMV6.1.0 upgraded from 5 on a Core2Duo

    mergerfs & snapraid

    Docker for urbackup, omada controller, adguard.

    Fixed it.


    TL:DR, my backups had 9M tiny files which was driving huge waste and growing the pairity.


    For those that might find this useful in the future, the saga is documented here.

    Mark D

    OMV6.1.0 upgraded from 5 on a Core2Duo

    mergerfs & snapraid

    Docker for urbackup, omada controller, adguard.

    • Offizieller Beitrag

    What backup program are you using? That info might be useful for forum users with a similar problem.
    ______________________________________

    I'm using ZFS for data storage and have used SNAPRAID / MergerFS , on a backup server, with a collection of dissimilar sized disks.


    For utility purposes, I use a separate dedicated hard drive (4TB) for client backups, Dockers, image files, etc.
    Where backups are concerned, I see the client drive itself as the first copy and the client backup as the second copy. Further protecting client backups themselves, with a second layer of backup or additional protection measures seems excessive, so it makes sense (to me) to put them on a utility disk. Using a separate disk has saved me from conflict issues between ZFS and dockers' overlayfs file system as well.


    In a similar manner, you might benefit from a utility disk.

Jetzt mitmachen!

Sie haben noch kein Benutzerkonto auf unserer Seite? Registrieren Sie sich kostenlos und nehmen Sie an unserer Community teil!

Benutzerkonto erstellen Anmelden