Setting up Incremental remote backups to OMV?

  • A long time Linux user as blue in NAS as possible. I started setting up OMV 2.2 2 weeks ago to replace a desktop "server", using a tiny Intel x86 netbook. Have Syncthing's working (plugin). Next comes incremental backup of the other machines all running GNU/Linux OSes.


    I do it from the desktop via rsnapshot & ssh atm. Pushes backups of the remote hosts daily, weekly and monthly to HDD "Uno" and "Deuzio" that are swaped every week. Swapped HDD being stored offsite.
    The few non-Linux OSes (Android and a Windows laptop) is sync'ed with Syncthing for our data; and with rsync or imaging for the systems confs.


    I'd like to hear those of you guys who do incremental data backup from remote hosts say a bit how do you manage that via OMV!?
    Before I start to set up rsnapshot to do auto incremental backups of the remote host the usual Linux way (setting up password-less key auth). As I may well be missing the NAS way, new to me as it is.

  • Okay, would like to hear how you guys push incremental backups from OMV. There might be a couple of plugins I overviewed.


    I added rsnapshot-once to rsnapshot. Now enjoying automatic incremental backup of all my personal non audio-video files as well as users and systems configs (~20 gig) from a desktop, a laptop and OMV itself. It's set up so that any new/changed content on any machines will be backed up at least once a day, provided the box's ON a couple of hours.
    As for restoration I tried NFS share. Wasn't smooth so I've set up sshfs on the external boxes and it works fine.

  • Here is what I do:


    Th goal is to back up a remote server onto omv and keep daily snapshots of the backups


    Prepare key on OMV:

    • Open Web Gui and go to System -> Certificates and create a new ssh Certificate. The result should look like this:


    Prepare remote-server

    • Create a user remote-backup on the server
    • allow user remote-backup to execute rsync as root:

      Code
      cat <<EOF > /etc/sudoers.d/remote-backup
      # remote backup
      # allow user remote-backup to execue rsync as root
      remote-backup ALL= NOPASSWD:/usr/bin/rsync
      EOF


    • create a file /home/remote-backup/bin/validate-backup-cmd.sh


      This checks the remote command and only allows commands starting with rsync --server --sender to be executed


    • Allow execution of the new script via ssh:

      Code
      mkdir ~/.ssh
      chmod 700 ~/.ssh
      nano ~/.ssh/authorized_keys

      and
      edit this file (inserting the pblic key you previously created)

      Code
      command="/home/remote-backup/bin/validate-backup-cmd.sh" ssh-rsa <some public key>


      This will only allow remote-backup to execute the shell script /home/remote-backup/bin/validate-backup-cmd.sh and nothing else

    • Make sure the file ~/.ssh/authorized_keys is only readable by its owner, otherwise you will get indeciperable errors later:

      Code
      chmod 600 ~/.ssh/authorized_keys


    On OMV:

    • Go to Services -> rsync and create a new rsync Job


      Extra options is: --numeric-ids -q -e '/usr/bin/ssh -oStrictHostKeyChecking=no -p 22 -i /var/lib/openmediavault/ssh/keys/openmediavault-1c69f786-25aa-4941-b5b6-4ef49514ec26' --delete-excluded --exclude=/dev/* --exclude=/proc/* --exclude=/sys/* --exclude=/run/* --exclude=/lost+found --hard-links


      consult a rsync man page for the rsync options. The -e '...' tells rsync to use ssh as transport not rsync. The options given to ssh are

      • -oStrictHostKeyChecking=no Accept the remote servers key without prompting
      • -p 22 Use port 22 (default)
      • [-i /var/lib/openmediavault/ssh/keys/openmediavault-1c69f786-25aa-4941-b5b6-4ef49514ec26] use the key created before
        (might be a bit tricky to find it, but it is in the given directory)
    • Add a rsnapshot job (Services -> Rsnapshot -> Add) to do daily incremental backups of the share above to some other share (e.g. snapshots)



    I am using this to back up several remote servers, but if you have a database running on one of them you should do a local backup before you rsync it.

  • Well, I followed through, but when I execute the job, I get this error:


    Code
    Please wait, syncing <remote-backup@openfiler299:/mnt/vg-sda2/sda2-shares/MusicOnHold> to </media/4547cf41-3d1b-4133-af1f-c0fd9312fc53/OpenFilerBackups/> ...
    edly closed (0 bytes received so far) [Receiver]
    rsync error: unexplained error (code 255) at io.c(605) [Receiver=3.0.9]


    I did manually run rsync and it was successful:

    Code
    root@omv-pd:/etc# rsync -avz -e ssh remote-backup@openfiler299:/mnt/vg-sda2/sda2-shares/MusicOnHold /media/4547cf41-3d1b-4133-af1f-c0fd9312fc53/OpenFilerBackups/
    remote-backup@openfiler299's password:
    receiving incremental file list
    MusicOnHold/
    MusicOnHold/ohm1106001.nch
    sent 34 bytes received 1195197 bytes 341494.57 bytes/sec
    total size is 1215886 speedup is 1.02


    Now what?


    Also, I'm not sure where the /home/remote-backup/bin/validate-backup-cmd.sh command is supposed to come into play?


    How might I manually run the rsync command with the extra options to verify the command line? When I try that, it's prompting for a password, which would imply I don't have the keys set up right?

    Code
    root@omv-pd:/var/lib/openmediavault/ssh/keys# rsync -avz -e ssh --numeric-ids -q -e '/usr/bin/ssh -oStrictHostKeyChecking=no -p 22 -i /var/lib/openmediavault/ssh/keys/openmediavault-1a5daba4-f676-4f33-a1b1-a7840deecf39' --delete-excluded --exclude=/lost+found --hard-links remote-backup@openfiler299:/mnt/vg-sda2/sda2-shares/MusicOnHold /media/4547cf41-3d1b-4133-af1f-c0fd9312fc53/OpenFilerBackups/
    remote-backup@openfiler299's password:


    sure enough, if I manually enter the password, rsync transfers files successfully....though no output. I guess I'm not understanding how the key exchange is supposed to work.

  • Not sure about the error from the remote side, but in you last command you do set the -e flag twice.
    Remove the first -e ssh and try again (and to debug, remove the -q flag (quiet)):

    Code
    rsync -avz --numeric-ids -e '/usr/bin/ssh -oStrictHostKeyChecking=no -p 22 -i /var/lib/openmediavault/ssh/keys/openmediavault-1a5daba4-f676-4f33-a1b1-a7840deecf39' --delete-excluded --exclude=/lost+found --hard-links remote-backup@openfiler299:/mnt/vg-sda2/sda2-shares/MusicOnHold /media/4547cf41-3d1b-4133-af1f-c0fd9312fc53/OpenFilerBackups/
  • I got that command from as revealed from OMV's cron job.I set up key exchange for root user, and changed the job to login as root and removed the extra options and it works as expected.


    I'm sure now that the problem is with trying to ssh as my remote-backup user...and I'm prompted for a password.


    I even tried this:


    and still getting prompted for remote-backup's password

  • I did not exactly what you did.


    can you execute

    Code
    ssh -i /var/lib/openmediavault/ssh/keys/openmediavault-1a5daba4-f676-4f33-a1b1-a7840deecf39 remote-backup@openfiler299


    -i is the path to the _private_ key file of the user trying to log in.


    The result should be an error message about your command not beeing allowed.

  • Hi,



    great !


    it sounds that it"s exactly what i'm searching;


    But need more explanations.


    datruche, you mentioned using rsnapshot. my goal is to backup my raspberry on my nas. So it could be possible with that ?


    Zoki, i tried your solutin, it works except the line



    Code
    command="/home/remote-backup/bin/validate-backup-cmd.sh" ssh-rsa <some public key>

    Sorry i d'ont know how use it ... Do i have to replace all the "ssh-rsa <some public key>" by my key ?



    Finally i wonder what is the best solution ?


    using rsync or rsnapshot with OMV?


    Thanks for your help,
    JD

  • hi,


    i continue to test.
    rsnapshot seems best way for me. Unfortunately, it can't be made using the OMV web interface. Am i wrong ?


    It seems that rsnapshot on omv can only backup shared folder but none remote folder.


    Do you confirm ?

  • (Note: a 3.X version of OMV is required to use the UrBackup plug-in.)
    _____________________________________________________________________


    I've been using Rsync to backup the data directories of a file server. The way I'm doing it uses the "Remote Mount" plugin, so the remote shares appear to be local to OVM.



    After sharing the mount points created above, Rsync is set up below.



    Under source, "Backups_R" means "remote". The destination is a share on the data drive of OMV. Videos_THPC is another client where a good number of large occasionally changing files reside.
    _______________________________________________________________________________


    I've been using UrBackup for a couple clients, for both full and incremental image and file backups. It works fine. However, since I'm using a Raspberry PI for both operations, for file server and client backup, I limit concurrent "UrBackup" backups to one as a time.



    With a 100mbs interface and an Arm processor, avoiding parallel operations whenever possible just makes sense. On the other hand, it's a very low cost, low power solution (about 12 watts) where I have cloned OMV boot media backups and hardware replacements in hand.


    With a few minor bumps to navigate, a NOOB (like myself) can do this.

Participate now!

Don’t have an account yet? Register yourself now and be a part of our community!