Backup to multiple disks

  • I need some advice on how to set up my backup.
    I have an internal data disk (6 TB) that I would like to back up regularly to two backup disks (3 TB each). Mirroring is enough, I don't need a history. The backup disks are installed in hot-swap trays and are stored elsewhere when not in use.


    I would prefer to just slide the trays in to start an automatic backup. When it's done, I get an email and just pull out the disks.
    I was looking for a ready-made solution, but I couldn't find one. The USB backup plugin can only be used for backups to one single hard disk. So I'll probably have to write a script.


    Problem 1)
    I don't want to manually split the internal hard disk data between the two external hard disks. I was thinking about using the Unionfilesystems plugin to merge the external disks to one file system. Are there better alternatives (raid)?


    Problem 2)
    Hot plug. On hardware level, the hot-plug already works, I can plug in and use the disks during operation. How can the hard drives be safely removed? Just pulling it out is certainly not a good idea neither for a raid nor for a unionfs. So how do I unmount them?


    Problem 3)
    The simplest part: copying the data, I would use rsync for this.

  • The USB backup plugin can only be used for backups to one single hard disk.

    That is not entirely true. You can backup different shared folders to different USB or eSATA disks. My ZFS pool is divided in several shared folders. For every shared folder I have created a separate USB backup job which uses two different external disks, one USB disk and one eSATA disk. Everytime I connect a disk only the assigned backup jobs are executed.


    You can do this in the same matter with your disks on hot swap tray.

    I don't want to manually split the internal hard disk data between the two external hard disks.

    It depends how many data you have on that internal drive. If you have only one share with all your data and the size exceeds to capacity of the external drive then you have a problem and then you can´t use the USB backup plugin.



    How can the hard drives be safely removed?

    The USB backup mounts the drive before starting the backup and unmounts it at the end. So you are safe to remove.



    I would use rsync for this.

    USB backup uses rsync too.

    OMV 3.0.100 (Gray style)

    ASRock Rack C2550D4I C0-stepping - 16GB ECC - 6x WD RED 3TB (ZFS 2x3 Striped RaidZ1) - Fractal Design Node 304 -

    3x WD80EMAZ Snapraid / MergerFS-pool via eSATA - 4-Bay ICYCube MB561U3S-4S with fan-mod

  • At the moment the 6TB disk is just filled up half, but this will change. Most data will be a single share containing lots of subfolders which i dont want to split up.


    Maybe I could somehow use symlinks to spread those subfolders into 2 backup folders and use USB backup to backup the subfolders. But I would have to watch the folder size permanently...

  • I would use the 2x3 TB as for storage (with the union fs plugin) and back then up to the single 6TB. That would be less plugging and unplugging and you would only need to carry one drive to your backup location.
    Also makes things easier since the 2x3 TB would be seen as a single drive.

  • lol, sometimes the simplest solution is also the most obvious. :D


    However, this might lead to another problem: The 6 TB disk is encrypted with luks (I didn't mention that before :/ ) and I unlock it manually via web interface after reboot.
    If I encrypt the 2x3 TB disks with luks and merge them with unionfs, wouldn't that be almost the same szenario as with hot-swapping?


    I didn't have time to dig in that direction on the weekend, but I'll look further into the unionfs/mergerfs... There must be a way to mount/unmount the overlay filesystem.

  • I'm worry about the short period of time when one disk is unlocked and the other not. That's the same szenario as if one tray is inserted/running and the other not. Mergerfs has to deal with this flawlessly.
    I ran some quick tests yesterday:
    I created and mounted two partitions sdc1 and sdc2. Then i created a overlay filesystem (fs1) from the mounts and copied some data on it. Afterwards I unmounted and remounted sdc1 and/or sdc2 serveral times and watched what happend to fs1. Aaaaand..... it happened what I expected only in the best case. Exactly all data of the mounted partition(s) were visible in the second it was mounted. I didn't check for any errors though.


    Additionally I could use "umount" to unmount fs1. Not sure if thats enought for a secure removal, but the readme says mergerfs does not use any caching.






    So maybe I was worried about nothing.... The shiny world of linux is new to me :/



    I'll try to make up a selfmade backup skript.



    On last thought on using 2x3 TB as internal disks with mergerfs... there seems to be some issued with certain database software i.e. plex. I didn't understand it completely, but may it's better not to do that.

  • I'm making progress...


    1) for unmouting the overlay filesystem the command

    Code
    fusermount -u /mountpoint

    can be used (see http://manpages.ubuntu.com/man…ty/man1/fusermount.1.html). I don't know if makes any difference to using "umount". I guess it doesn't make any difference, since I can hardly find any information about unmouting mergerfs or fuse filesystems.


    2) A script for the whole process of mounting and backup can be started via "udev" rule. See: https://brokkr.net/2017/06/12/…iling-and-how-i-fixed-it/

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!